Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Hal Cain]

[It appears that my recent message was empty when sent. I apoligize. What I 
attempted to send follows]
On Mon, 7 Nov 2011 11:05:14 +0100, Bernhard Eversberg  
wrote:

>07.11.2011 10:55, Jim Weinheimer:
>> What then
>> will be the purpose of ISO2709 except one: to transfer a catalog record
>> from one library catalog to another?
>>
>I know of no other purpose. But be that as it may, my point is that
>even for this function, it is no longer technically necessary.
>For all intents and purposes, MARC may live on forever without
>the need to deal with ISO2709. It is technically obsolete, but we
>need not care.
>Can anyone please prove me fundamentally wrong, or confirm what I say?

According to my small experiences in the years before I used modern library 
systems, when I was trying to extract data from MARC files (e.g. to carry out 
authority control in systems that didn't have authority modules, before 
proceeding to print catalogue cards), raw MARC data in communications format 
was a beast, and I wasn't nearly programmer enough to write routines to 
deconstruct it!

However, once I began to see how competent systems handled MARC, it became 
plain that what they were doing was basically to create a matrix and populate 
it with the tag values, the indicator values, and the subfield data prefixed by 
the subfield code.  Then the indexing routines read the matrix (not the raw 
MARC ISO2709 data) and distributed the data into the appropriate areas of the 
system's internal table structure.  From those tables, I was able, when 
required, to obtain what I wanted by direct query on the appropriate part of 
the database. When it was necessary to export a single MARC record, a group of 
them, or indeed the whole database, the system had routines which reversed the 
process (and, last of all, counted the number of characters in order to fill in 
the record length element of the MARC leader). This was extremely burdensome to 
programmers who came to the game in the 1990s and had no background in early 
data processing, chiefly of text rather than numbers, but in its time it was 
pure genius. Nowadays it's a very special niche, and the foreignness to 
programmers and designers of the processes involved probably plays a part in 
keeping us from having really good cataloguing modules and public catalogues; 
and I can understand the frustration entailed for those who expect to 
interrogate a database directly.

Bear in mind, though, that using a modern cataloguing module (Horizon is the 
one I'm most familiar with), I can search for a record on a remote system, e.g. 
the LC catalog, through Z39.50, and have the record on my screen, in editable 
form, in a second or two, indistinguishable from a record in the local 
database. The system's internal routines download the record in MARC format 
(ISO 2709, hated by Jim) and build the matrix which feeds the screen display. 

The success of these operations depends on the consistency of the records 
available. The complexities arise from the complexities in the data we work 
with, the consistency of our markup (recorded in MARC tags and subfields, a 
number of which are very specific and don't occur very often, but are applied 
to specific purposes when they occur, e.g. 254) and sometimes the 
inconsistency, as when we don't distinguish parallel tiles or alternative 
titles from other title elements. Those inconsistencies are not so much caused 
by MARC, certainly not by MARC in ISO2709 format; they result from the belief 
that specific codes for those elements cannot be created because the coding 
structure is already full; which is only half true.

Now, if what you want is a data storage which can be queried and reprocessed 
directly, without going through conversions to and from ISO2709, that's 
something to discuss in its own right; but the tools to translate data between 
ISO2709 and tabular form do exist, and they operate "on the fly" within their 
own systems.  For working outside a library system, MarcEdit is one commonly 
used; for querying specific bibliographic elements it's far from simple.

But even Voyager, the ILS providing LC's catalog, supports searches within 
specific MARC fields and subfields of the LC bibliographic database.  

Maybe the problem is that there's no universal bibliographic database that 
isn't MARC-based? And therefore one has to deal with MARC with its 
inconsistencies and idiosyncracies? I have no solution to that problem; the 
weight of our history is a considerable obstacle when it comes to trying to do 
something different.

Hal Cain, who acknowledges his knowledge is incomplete and liable to correction
Melbourne, Australia
hegc...@gmail.com

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Hal Cain]

On Mon, 7 Nov 2011 11:05:14 +0100, Bernhard Eversberg  
wrote:

>07.11.2011 10:55, Jim Weinheimer:
>> What then
>> will be the purpose of ISO2709 except one: to transfer a catalog record
>> from one library catalog to another?
>>
>I know of no other purpose. But be that as it may, my point is that
>even for this function, it is no longer technically necessary.
>For all intents and purposes, MARC may live on forever without
>the need to deal with ISO2709. It is technically obsolete, but we
>need not care.
>Can anyone please prove me fundamentally wrong, or confirm what I say?

According to my small experiences in the years before I used modern library 
systems, when I was trying to extract data from MARC files (e.g. to carry out 
authority control in systems that didn't have authority modules, before 
proceeding to print catalogue cards), raw MARC data in communications format 
was a beast, and I wasn't nearly programmer enough to write routines to 
deconstruct it!

However, once I began to see how competent systems handled MARC, it became 
plain that what they were doing was basically to create a matrix and populate 
it with the tag values, the indicator values, and the subfield data prefixed by 
the subfield code.  Then the indexing routines read the matrix (not the raw 
MARC ISO2709 data) and distributed the data into the appropriate areas of the 
system's internal table structure.  From those tables, I was able, when 
required, to obtain what I wanted by direct query on the appropriate part of 
the database. When it was necessary to export a single MARC record, a group of 
them, or indeed the whole database, the system had routines which reversed the 
process (and, last of all, counted the number of characters in order to fill in 
the record length element of the MARC leader). This was extremely burdensome to 
programmers who came to the game in the 1990s and had no background in early 
data processing, chiefly of text rather than numbers, but in its time it was 
pure genius. Nowadays it's a very special niche, and the foreignness to 
programmers and designers of the processes involved probably plays a part in 
keeping us from having really good cataloguing modules and public catalogues; 
and I can understand the frustration entailed for those who expect to 
interrogate a database directly.

Bear in mind, though, that using a modern cataloguing module (Horizon is the 
one I'm most familiar with), I can search for a record on a remote system, e.g. 
the LC catalog, through Z39.50, and have the record on my screen, in editable 
form, in a second or two, indistinguishable from a record in the local 
database. The system's internal routines download the record in MARC format 
(ISO 2709, hated by Jim) and build the matrix which feeds the screen display. 

The success of these operations depends on the consistency of the records 
available. The complexities arise from the complexities in the data we work 
with, the consistency of our markup (recorded in MARC tags and subfields, a 
number of which are very specific and don't occur very often, but are applied 
to specific purposes when they occur, e.g. 254) and sometimes the 
inconsistency, as when we don't distinguish parallel tiles or alternative 
titles from other title elements. Those inconsistencies are not so much caused 
by MARC, certainly not by MARC in ISO2709 format; they result from the belief 
that specific codes for those elements cannot be created because the coding 
structure is already full; which is only half true.

Now, if what you want is a data storage which can be queried and reprocessed 
directly, without going through conversions to and from ISO2709, that's 
something to discuss in its own right; but the tools to translate data between 
ISO2709 and tabular form do exist, and they operate "on the fly" within their 
own systems.  For working outside a library system, MarcEdit is one commonly 
used; for querying specific bibliographic elements it's far from simple.

But even Voyager, the ILS providing LC's catalog, supports searches within 
specific MARC fields and subfields of the LC bibliographic database.  

Maybe the problem is that there's no universal bibliographic database that 
isn't MARC-based? And therefore one has to deal with MARC with its 
inconsistencies and idiosyncracies? I have no solution to that problem; the 
weight of our history is a considerable obstacle when it comes to trying to do 
something different.

Hal Cain, who acknowledges his knowledge is incomplete and liable to correction
Melbourne, Australia
hegc...@gmail.com

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Hal Cain]

On Mon, 7 Nov 2011 11:05:14 +0100, Bernhard Eversberg  
wrote:

>07.11.2011 10:55, Jim Weinheimer:
>> What then
>> will be the purpose of ISO2709 except one: to transfer a catalog record
>> from one library catalog to another?
>>
>I know of no other purpose. But be that as it may, my point is that
>even for this function, it is no longer technically necessary.
>For all intents and purposes, MARC may live on forever without
>the need to deal with ISO2709. It is technically obsolete, but we
>need not care.
>Can anyone please prove me fundamentally wrong, or confirm what I say?

According to my small experiences in the years before I used modern library 
systems, when I was trying to extract data from MARC files (e.g. to carry out 
authority control in systems that didn't have authority modules, before 
proceeding to print catalogue cards), raw MARC data in communications format 
was a beast, and I wasn't nearly programmer enough to write routines to 
deconstruct it!

However, once I began to see how competent systems handled MARC, it became 
plain that what they were doing was basically to create a matrix and populate 
it with the tag values, the indicator values, and the subfield data prefixed by 
the subfield code.  Then the indexing routines read the matrix (not the raw 
MARC ISO2709 data) and distributed the data into the appropriate areas of the 
system's internal table structure.  From those tables, I was able, when 
required, to obtain what I wanted by direct query on the appropriate part of 
the database. When it was necessary to export a single MARC record, a group of 
them, or indeed the whole database, the system had routines which reversed the 
process (and, last of all, counted the number of characters in order to fill in 
the record length element of the MARC leader). This was extremely burdensome to 
programmers who came to the game in the 1990s and had no background in early 
data processing, chiefly of text rather than numbers, but in its time it was 
pure genius. Nowadays it's a very special niche, and the foreignness to 
programmers and designers of the processes involved probably plays a part in 
keeping us from having really good cataloguing modules and public catalogues; 
and I can understand the frustration entailed for those who expect to 
interrogate a database directly.

Bear in mind, though, that using a modern cataloguing module (Horizon is the 
one I'm most familiar with), I can search for a record on a remote system, e.g. 
the LC catalog, through Z39.50, and have the record on my screen, in editable 
form, in a second or two, indistinguishable from a record in the local 
database. The system's internal routines download the record in MARC format 
(ISO 2709, hated by Jim) and build the matrix which feeds the screen display. 

The success of these operations depends on the consistency of the records 
available. The complexities arise from the complexities in the data we work 
with, the consistency of our markup (recorded in MARC tags and subfields, a 
number of which are very specific and don't occur very often, but are applied 
to specific purposes when they occur, e.g. 254) and sometimes the 
inconsistency, as when we don't distinguish parallel tiles or alternative 
titles from other title elements. Those inconsistencies are not so much caused 
by MARC, certainly not by MARC in ISO2709 format; they result from the belief 
that specific codes for those elements cannot be created because the coding 
structure is already full; which is only half true.

Now, if what you want is a data storage which can be queried and reprocessed 
directly, without going through conversions to and from ISO2709, that's 
something to discuss in its own right; but the tools to translate data between 
ISO2709 and tabular form do exist, and they operate "on the fly" within their 
own systems.  For working outside a library system, MarcEdit is one commonly 
used; for querying specific bibliographic elements it's far from simple.

But even Voyager, the ILS providing LC's catalog, supports searches within 
specific MARC fields and subfields of the LC bibliographic database.  

Maybe the problem is that there's no universal bibliographic database that 
isn't MARC-based? And therefore one has to deal with MARC with its 
inconsistencies and idiosyncracies? I have no solution to that problem; the 
weight of our history is a considerable obstacle when it comes to trying to do 
something different.

Hal Cain, who acknowledges his knowledge is incomplete and liable to correction
Melbourne, Australia
hegc...@gmail.com

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[J. McRee Elrod]

Bernhard said:

>I know of no other purpose. But be that as it may, my point is that
>even for this function, it is no longer technically necessary.
>For all intents and purposes, MARC may live on forever without
>the need to deal with ISO2709. It is technically obsolete, but we
>need not care.

I would agree with Bernhard.  SLC sends MARC records to about 50
clients, including twelve electronic publishers and aggregators,
using a variety of mean: loading records to their server, having them
download from our server, sending as a single .mrc file, and sending
as individual named files (often the e-ISBN).  Our programmer never
*heard* of ISO2709.  Just once we were asked about it by a Chinese
client, and had to look it up to see what they were talking about.

We began in 1979 using UTLASMARC, switched  1990 to USMARC, and 1999
to MARC21, but still keeping a couple of features* we like from
UTLASMARC.  We've never even looked at ISO2709.  

Is it the basis of Z39.50?  The variations we discover in Z39.50
searching are, I assume, features of individual systems, e.g., not all
access keys defined by all systems, and the same key is defined
differently; in some catalogues the title search is exact, and in some
it is keyword,  returning a raft of irrelevant hits.  Searching is I
assume outside ISO2709, since there is *no* standardization in that
area.


   __   __   J. McRee (Mac) Elrod (m...@slc.bc.ca)
  {__  |   / Special Libraries Cataloguing   HTTP://www.slc.bc.ca/
  ___} |__ \__



*Our favourite UTLASMARC retained feature is the single 090 per
library, far easier than MARC21's one 852 per item.  For example:

090 0  $aAB/1234.5/C678/2011$bLibrary$ccopies$dvolumes $ebar code or
accession number$fsublocation$rnotes.  There are other less used
subfields.  

Our print programs produce the correct number of labels and cards
based on the 090, e.g., $c1-2$d1-2 produces four labels.  The "/"
produces a line break on a label or card.  If one wants a forward slash
in the print product, e.g. a split year, one keys "\".

This 090 is translated into multiple 852s on export for those which
wish them.  Several Canadian library clients still use UTLASMARC 090.

All this without aid of any ISO so far as we know.

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Jim Weinheimer]

On Mon, Nov 7, 2011 at 11:05 AM, Bernhard Eversberg wrote:


> But be that as it may, my point is that
> even for this function, it is no longer technically necessary.
> For all intents and purposes, MARC may live on forever without
> the need to deal with ISO2709. It is technically obsolete, but we
> need not care.


Perhaps it will live on as one developer described, when last week at lunch
we were discussing the "old days" of the ISO2709 format for AGRIN3 data
that he (and I and everybody) had to work with before we all changed it to
XML.

He mentioned that he keeps the specifications in a drawer of his desk as a
momento mori. Once in awhile he takes them out just to gaze upon and to
remind himself of other realities!

-- 

James L. Weinheimer  weinheimer.ji...@gmail.com
First Thus: http://catalogingmatters.blogspot.com/
Cooperative Cataloging Rules: http://sites.google.com/site/opencatalogingrules

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Bernhard Eversberg]

07.11.2011 10:55, Jim Weinheimer:




With ISO2709, it is designed to transfer a complete catalog record
from one catalog into another catalog.

Yes, but Web services on any MARC based catalog need not suffer
from that, Web services can be constructed without paying any attention
to the ISO structure. I said that much in my post. It is regrettable
that up until now we still have not many useful web services as part
of library OPACs. But the reason for this is certainly not ISO2709.



Have you ever seen or heard of a web service based on ISO2709?

No, but there is, logically, no need to deal with ISO in order to
construct web services. Any technical needs can be eliminated and
should have been long ago.


What then
will be the purpose of ISO2709 except one: to transfer a catalog record
from one library catalog to another?


I know of no other purpose. But be that as it may, my point is that
even for this function, it is no longer technically necessary.
For all intents and purposes, MARC may live on forever without
the need to deal with ISO2709. It is technically obsolete, but we
need not care.
Can anyone please prove me fundamentally wrong, or confirm what I say?

B.E.

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Jim Weinheimer]

On Mon, Nov 7, 2011 at 10:21 AM, Bernhard Eversberg wrote:


> Jim, my point is, in nuce:
>   "Yes, MARC is horrible, but ISO is not the reason".
>
> You wrote:
>
>>
>> With ISO2709, it is designed to transfer a complete catalog record
>> from one catalog into another catalog.
>>
>

>  Yes, but Web services on any MARC based catalog need not suffer
> from that, Web services can be constructed without paying any attention
> to the ISO structure. I said that much in my post. It is regrettable
> that up until now we still have not many useful web services as part
> of library OPACs. But the reason for this is certainly not ISO2709.
>


Have you ever seen or heard of a web service based on ISO2709? What then
will be the purpose of ISO2709 except one: to transfer a catalog record
from one library catalog to another?

But this now appears to be the second aspect of MARC, which is what most of
the discussion is about, not about ISO2709 itself, but the coding, e.g.
100b 300c and so on. In one sense, this is much less of a problem because
we are talking about mere computer codes, and those codes can display
however someone wants them to display.

So, when developers say that they don't like MARCXML, this is a lot of what
they are talking about since they want and expect the coding to say "title"
and "date of publication" and they don't want to look up what 245a or 300c
means. (There are also the codes that must be dug out of the fixed fields
such as the type of dates and dates in the 008, the language code, etc. but
that is yet another matter)

Of course, we run into the problem of library jargon here, since 245a is
not "title" but "title proper" and not only that, it includes the
"alternative title" plus it includes individual titles when an item lacks a
collective title. There may be some more nuances as well. Therefore, 245a
implies separate access to a lot of other types of titles. Non-cataloger
developers cannot be expected to know or understand any of this. So, if the
format codes it , that is misleading, while coding it as
, developers will just think it's a weird name for a title.

This is complicated and at the moment I don't know how it can be solved.
Perhaps our traditional library distinctions will disappear in the new
environment, but I hope not.

-- 

James L. Weinheimer  weinheimer.ji...@gmail.com
First Thus: http://catalogingmatters.blogspot.com/
Cooperative Cataloging Rules: http://sites.google.com/site/opencatalogingrules

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Bernhard Eversberg]

Jim, my point is, in nuce:
   "Yes, MARC is horrible, but ISO is not the reason".

You wrote:


I wish that were true. ISO2709 is the standard way libraries exchange
 their records, and this means that anybody who wants library
information must work with ISO2709. ISO2709 was designed to make
catalog cards,

No, even worse: MARC itself was designed for that as its primary
function. ISO, for all its vices, does not enforce that kind of
restriction.



With ISO2709, it is designed to transfer a complete catalog record
from one catalog into another catalog.

Yes, but Web services on any MARC based catalog need not suffer
from that, Web services can be constructed without paying any attention
to the ISO structure. I said that much in my post. It is regrettable
that up until now we still have not many useful web services as part
of library OPACs. But the reason for this is certainly not ISO2709.

Can someone with more MARC insight than me please confirm this so we
can finally put this matter to rest?


B.Eversberg

Re: [RDA-L] Offlist reactions to the LC Bibliographic Framework statement

[Jim Weinheimer]

On Mon, Nov 7, 2011 at 8:00 AM, Bernhard Eversberg wrote:


> Jim, ISO2709 is a nuisance, agreed. And I dislike it no less than you
> do because I'm a real programmer and know what it feels like.
> But don't let's get carried away and rush to premature conclusions with
> inappropriate metaphors. Rather, consider this:
> Would you tear down your house and rebuild it from the ground up
> if the old wallpaper gives you the creeps?
>
> For that's what ISO2709 is: mere wallpaper. Easily replaced or painted
> over. Nothing serious, nothing that affects any qualities of the building.
>


I wish that were true. ISO2709 is the standard way libraries exchange their
records, and this means that anybody who wants library information must
work with ISO2709. ISO2709 was designed to make catalog cards, and that is
what it still does today, only the cards are not printed on card stock,
they are printed on the computer screen. Certainly, they can be searched in
ways different from a card catalog, but this is because of the mere fact
that they reside in the computer--not because the format is any more
amenable to searching.

Today, most web developers I know do not want to copy and reformat and
maintain duplicates of records that are on different systems.
They want much more to interoperate with them, and they can do this through
various APIs. For instance, I can add a Google Books API that will
search--in the background--Google Books in all kinds of ways and return one
record, or multiple records. It does not give me the entire Google metadata
record, nor do I want it. (as ISO2709 does--by definition) I want to work
with the Google metadata *on the fly* so that I do not have the
responsibility to keep the record current, reformat it and have to do all
kinds of additional work. Keeping the record current is Google's
responsibility--not mine and I shouldn't have to do it.

With ISO2709, it is designed to transfer a complete catalog record from one
catalog into another catalog. It is not designed for interactivity. Here is
a practical example. At LC, they have lots of sets of records where you can
interact with them http://memory.loc.gov/ammem/oamh/oai_request.html. So, I
could have a local catalog on e.g. dance, and I could search behind the
scenes--if I set the machine correctly--the records for LC's dance
instruction manuals. I can display these records as I wish because they are
in XML. I would not have to download all records in ISO2709, convert them
in MARCEdit, put them into my own database, where the URLs and other
information may change in the future, since potentially it is a ton of work
to maintain records for materials on the web.

Another example is the Worldcat Search API
http://www.worldcat.org/affiliate/tools?atype=wcapi. There is no mention of
ISO2709 there. Plus, I implemented the Worldcat Citations API when I was at
AUR:
http://www.oclc.org/developer/documentation/worldcat-search-api/formatted-citations
and an example:
http://www.galileo.aur.it/cgi-bin/koha/opac-detail.pl?bib=24135. In the
right-hand column, you will see "Get a Citation". When you click it, you
will see citation formats (in XML, not ISO2709) taken on the fly from
Worldcat and reformatted by the system I created. This is a simple example
and matters could become much more complex, if someone desired.

The fact is, most developers want to work with APIs in these kinds of ways
instead of having to download, convert (mostly an extremely difficult job
to come out with anything coherent), upload into your own system, and then
maintain those records. That is horribly inefficient, and unnecessary,
today.

Why don't more developers work with library metadata? To me, the answer is
absolutely obvious. We are not making APIs that developers want to work
with, and one reason is that we keep maintaining that if somebody wants our
information bad enough, it is "easy" to work with ISO2709 records by
downloading, reformatting, etc. but that is wrong. Working with APIs is
what is easy and if you use ISO2709 you absolutely cannot do that.

Developers don't want--or need--to jump through all of those hoops when
they don't have to, and they prefer to work with other systems. So they
don't use our records and prefer, e.g. Amazon, which has all kinds of APIs.

Unfortunate. But perhaps it is something that the Bibliographic Framework
will address and our metadata will be more usable in the information
universe.

-- 

James L. Weinheimer  weinheimer.ji...@gmail.com
First Thus: http://catalogingmatters.blogspot.com/
Cooperative Cataloging Rules: http://sites.google.com/site/opencatalogingrules