Next Generation Data Format - Eric Lease Morgan - University of Nortre Dame - May 7, 2008

Eric Lease Morgan started with "its not about 'what we do', it is about 'how we do it'."

How we got to where we are

Morgan reminded us that we have been using services (LC cards, OCLC, etc.) to help us build catalogs for generations and using the MARC record since 1965. It is a sequential specific data structure. Then, the invasion of Dialogue, CD-ROMS, network connections, and jump ahead to today, where Information structures, such as Google, are based on different indexing principles.

Redefine the definition of librarianship

Traditional core mission of librarianship is to collect, organize, archive, and disseminate data, information, and knowledge for our respective communities.

Here's what Morgan suggests:

Reduce dependence

Librarians should have more control. They should find ways to use technology to provide services. Using Open Source Software is one way.

Exploit technology

Morgan advocates using relational databases, indexing systems (Googlebox, Keynote Search, etc.), XML.

If we want to communicate to other places, in other words beyond the library community, we must speak other languages, and XML is the language being used.

As a "step in the right direction", conversion programs can help change MARC to MARC XML. The idea of describing a work is valid, but the method does not work in the broader world. The "Secret Codebook", better known as MARC, is limited because you need the code to interpret the fields.

A word from the audience (Dodie Gaudet) mentioned that the tags in XML are now in English, but MARC using numbers is more universal.

Commercial products use indexers to create interface. For example: Koha uses Zebra; Aqua developed a proprietary Indexer.

Work collaboratively

Work with peers and stakeholders in the inside and outside the library world.

"Next generation" library catalogs

Provide service not just catalog systems. As a part of the community, use this base to create services at the local level. Add, annotate, cite, compare and contrast, plot a map, find similar, trace authors, etc.

In response to a question, Morgan suggested see the applications (Scriblio, WordPress, VuFind, etc.) demonstrated during the pre-conference (MLA May 6th).

Future of MARC: Challenges and opportunities of 21st century cataloging

Dr. William E. Moen, SLIS, Univ. of North Texas

Presentation will be available on the MLA website after the conference.

We'd better be showing our value to our users. What services do we need to provide in order to keep us from becomming marginalized.

Focus on the methods - what we don't want to do is to expose our really complex structures to our users. Don't need to know - we will marginalize ourselves even more if we insist on telling out users how these systems work.

Discussion is only partially about MARC

broader digital info landscape
oops - slide changed!

Catalogs are limited in light of all sorts of other resource discovery tools that are out there. if we don't accept this, we are not placing oursleves effectively in the market.

What do we mean when we say MARC?
>>>structural elements of the format - this may go away in favor of mark up language

>>>Metadata scheme as defined by MARC 21
Fields, subfields, indicators and their semantics.

Approaching MARC's future
>requirements for a record format / metatadata scheme

>Resonding to recent developments

>Looking at empirical data.

Thinking about requirements

what metadata scheme is appropriate? Granularity - lossless data mapping - don't want to lose the finer shades of meaning intrinsic to the original data. XML will be the basis for this
Transparency

McCallum/s 10 format attribures of MARC forward
XML
Granularity
Versatility
Extensibility
Modularity
Hierarch support
Crosswalks
Tools
Cooperative management
Pervasive

AT ALA 2007 there will be a program on furture of marc

Recent developments
FRBR - conceptural model for the bibliographic universe, according to Barbara Tillett
The aim of the frbr study was to produce a framework that would provide a clear, precistly stated understanding of bibliogrpahic info.

frbr model
entity-relationship modeling

what is it that the bib record aims to be about
User tasks - what is it that we expect the record to achieve in terms of answering user needs?
Find? Identify? Select? Obtain?

frbr is having an amazing impact on catalogs - new terminology and conceptural modes incorporated in
RDA
Statement on cataloging principles

Assisting in understandning better the range of relationships in the bib universe

Implementation in catalogs to improve user experience.

Next generation of description
RDA - Resouce description and access
Focus on guidelines fo rcontent creation
Separation from syntax or record format

move in RDA to be totally separate from record format.

Grassroots effort to address next generation loibrary catalog and data format (wikis for example)

Key development - use of the term metadata. how do we link up with other sources of metadata, commercial, museum descriptions, etc. We need to acknowledge that we don't have enough catalogers in the world to mark up everything that is good out there. We need to learn to accept metadata from other sources.

What do catalogers actually do with the MARC record?
a record is an artifact that can be examined. it's a result of the cataloging enterprise and includes loads of decisions that were made as to history of description in your library.
ou follow rules, standards, encode it in a MARC structure. MARC is very rich; it has over 2000 data structures. How do we use all of these? Is the richness exploited? There are about 500 places within MARC that can hold author/title info!

Did an analyis - 4% of all fields used accounted for 80% of occurences
96% of all fields acount for only 20% of occurrences

Dr.Moen did a project called MCDU (MARC content designation utilization)
Looked at the artifiact created and look at what is used, not used, how fields are used. Used the entire world cat database of over 56 million records. Set up a procedure to take apart the bib records in a mySQL databaswe. Needed to break down the records so they could meaningfully look at all parts of the records. Divided into LC records and non LC records. Separated out records by item type because some MARC tags are relevant only to the type of material they dscribe. See the slide presentation on the MLA website for details on the findings of this study.
Lots and lots of fields not used regulary. 16 fields are used commonly across all 10 formats. they are the usual ones 020, 050, 082, 100, 245, etc. Use this info to develop a core of kinds of info that is the most useful

Trying to conceptualize a change in trends of utilization

This report will be published within the next couple of months.

To consider -whatis needed in a bib record? What is needed to support user tasks? Management of info resources? How do systems use the infrequently used data. What do we do about the 62% of all fields used in less that 1% of all records in OCLC WorldCat? Coding has to be done to prepare for these little used fields.

We need to rethink MARC in light of value? How can we justify the high cost of MARC? What is a 'real good' MARC record? How can we deal with the shrinking budgets if we are focus on this highly priced format? There are lots of metadata schemes; MARC won't be one of the choices!

Questions

What is the next step in the decision making process - in that 62% of fields used in 1% of records - who will decide if these should be kept or not? Who will make the decision?

Dr.Moen is doing these presentations, trying to get the info out to the community and the cataloging rules people. Start by looking at your own cataloging practices. To what extent do you use certain fields. Hopes for an open discussion even thoough we might not have a vote on the national level. Dr. Moen would like to 'freeze marc' right at this point so it can be further examined.

Question - can you map your commonly used fields to dublin core

Yes, can be done.

Future of MARC -- Dr. Bill Moen

Future of MARC: the Challenges and Opportunities of 21st Century Cataloging
[Eventually, the link to this presentation will be on the MLA Conference website]

• "We need to be adding value through our practices....We'd better be saving people time and money."
• "We need to be meeting the needs of our users.
• Less focus on the methods. Our methods should be invisible and unobstructive. How can we take our structures and hide them, but not hide the power that they provide?

There's nothing wrong with thinking in terms of market share -- that's our reality. We no longer have a lock on the target market that we used to. We have a limited set of resources that are availble through our catalog, and now users can see how limited they are compared with everything else that's out there.

What do we mean when we say "MARC"?
Record format, as defined by ISO 2709/ANSIZ39.2, & structural elements of the format -- this is going to go away and be replaced by markup languages
Metadata scheme -- defined by MARC 21 and fields, subfields, indicators and their semantics

Approaching MARC's future:

Requirements for a record format/metadata scheme
Goldsmith & Knudson's Requirements:
Granularity -- how fine a detail can you get to
Transparency --
Extensibility --

Roy Tennant's Requirements (slide went by too fast)

McCallum's 10 Format Attributes

• XML
• Granularity
• Versatility
• Extensibility
• Modularity
• Hierarchy support
• Crosswalks
• Tools
• Cooperative Management
• Pervasive

Functional Requirements for Bibliographic Records (FRBR)
Produce a framework that would provide a clear, precisely stated and commonly shared understanding of what it is that the bib record aims to provide information about and what it is that we expect the record to achieve in terms of answering user needs.
Based on Entity-Relationship modeling (work, expression, manifestation, item / persons, corporate bodies)
An important part of the FRBR report was the focus on users & user tasks: find, identify, select, obtain
If bib records are not supporting user tasks, what's the point?

FRBR is introducing new vocabulary and a new understanding of the items we catalog. We're seeing the implementation of these ideas in new library catalogs.
We currently ask our users to put up with a lot of noise, lots of individual record, rather than letting them actively drill down with limited choices

Responding to recent developments
No AARC3; Resource Description and Access (RDA), which are more guidelines on content creation, and have separation from syntax or record format

Library Systems and Data Formats (wiki) -- grassroots efforts to look at

Metadata

• Essential in library applications
• Variety of metadata schemes
• Variety of functions and services supported
• Increasing use of machine-generated metadata - there aren't enough catalogers in the world
• Role of handcrafted metadata needs continuing review & assessment

These are not threats to the livelihood of catalogers/TS librarians. There's plenty of work around, but they have to change the approach to handcrafted metadata -- where's the value added of that hands-on work.

Looking at empirical data
The cataloging record you create is an artifact that reflects decisions, policies and choices, and can be investigated to see patterns and needs.
Catalogers create metadata that can be very rich (MARC).

There had never been a study before of exactly how catalogs actually construct MARC records and what they actually do. So, Dr. Moen did one.
There is a *lot* of redundancy in the records. The 80/20 rule holds: 4% of fields/subfields accounted for 80% of occurrences, 96% of all fields accounted for 20% occurrences (where occurrence = data in the field)

MCDU Project -- Reports containing results of analysis of utilization, commonly used elements
OCLC gave them the entire WorldCat as of May 2005, so they're working from the whole dataset
82 hours for a script to process and load the records as MySQL and 258 GB = mad data set
Millions of book records
Categories of Questions: General profile of the dataset & actual numbers of occurrences
167 fields used
14 fields accounted for 80% of all occurrences
21 fields accounted for 90% of all occurrences
110 fields occur in less than 1% of all records
"656 a:" occurred in 1 record of out 7.5 million. Why?

They also looked at field/subfield combinations. They keep finding a small core of elements that are commonly used -- are these what catalogers should be focusing on? Is this what the machine-generated cataloging should be focusing on?

Making Sense of Numbers
Not interpreting the value of an individual fields, but looking at patterns and larger recommendations/guidelines. Comparing the data to FRBR user tasks.Is there a common core of elements that are used? Is there a threshold below which things just aren't used so much?

Are library catalogers providing data to support FRBR tasks?
In MCDU dataset, only 59 fields/subfields (13% of total) occur at or above the threshold of use in OCLC book records.

Questions for consideration

• What is needed in a bib record? Are catalogers working too hard and creating stuff no one uses?
• Support for four user tasks? In the context of FRBR, what does it mean to support a user task?
• How can we use metadata for effective management of information resources?
• How do your systems use the infrequently used data? What about the 62% of all fields used in less than 1% of records?
• Can we argue persuasively for the cost/benefit for your existing practice?
• Should the focus be on the high-value, high-impact, high-quality data in a few fields/subfields? Can we identify these? What would it mean to costs of cataloging to focus this way? What would this mean for training new catalogers?
• Can MCDU results inform your local practices?
• What metadata scheme will we use? It won't be MARC.
• [missed one]

Confluence of change -- all of this data and the realities of life now mean that change will happen. It's just a question of when and will we be prepared?

Is the next study a look at which fields users are using? Is this useful to know? What data about how users search is useful? There have been a lot of studies done of how users search in the Search Engine world; how do these translate into the library field?

[Jenn sez: There are a lot of stunned heads in this audience. As a non-cataloger, I'm hoping this hasn't just caused heart attacks throughout the room.]