Hi Karen. I have done some playing with the load process. I think you failed to set a 'start record' number in marc2bre.... once you have any records in the database, you need to determine the next start number, and pass it to marc2bre... for example,
perl marc2bre.pl -startid=2017 ebooks/gale.mrc | perl direct_ingest.pl -idler 0 | perl pg_loader.pl -or bre -or mrd -or mfr -or mtfe -or mafe -or msfe -or mkfe -or msefe -a mrd -a mfr -a mtfe -a mafe -a msfe -a mkfe -a msefe -idler 0 > gale.sql Hmm... That reminds me... in order to make the output of such piped commands legible, I modified direct_ingest and pg-loader to not put out the record count/records per second stuff that each program normally does, hence the "-idler 0" in the above command. I guess I'd better get my head back in this space long enough to submit a patch... With existing code, use something like: perl marc2bre.pl -startid=2017 ebooks/gale.mrc | perl direct_ingest.pl | perl pg_loader.pl -or bre -or mrd -or mfr -or mtfe -or mafe -or msfe -or mkfe -or msefe -a mrd -a mfr -a mtfe -a mafe -a msfe -a mkfe -a msefe > gale.sql don (not mcmorris, and not dan, either) >>> [EMAIL PROTECTED] 02-Aug-2007 12:02 PM >>> I went through the .marc file and found the records containing some of the strange characters, and went back and looked at those records in Horizon. They display as diamond shapes with question marks there, and the odd capitalization you mentioned shows up there as well. Our cataloger is looking at those records, intending to correct them. Meanwhile, I tried using a smaller set of MARC records that didn't include any of the records I had problems with in the .marc to .bre phase. Everything went smoothly with no apparent errors until I got to: psql -U evergreen evergreen evergreen# \i ~/kent1000records.sql; and I got a whole series of these guys again: ----- Original Message ----- From: Joe Atzberger <[EMAIL PROTECTED]> To: [email protected] Sent: Wed, 1 Aug 2007 17:24:36 -0400 Subject: Re: [OPEN-ILS-DEV] Problem Importing MARC records > Karen -- > > Is there something strange going on with your character set (and > capitalizations)? In your records, what are the > characters that display in the error log (or at least in my email) as squares? > Perhaps they are illegal escape > characters or something more appropriate in UTF-8. > > Is is possible your line endings were reformatted or scripts randomly forced to > uppercase? To me, "nEW yORK" looks like > the result of a tr/[A-z]/a-Z/ and "\N" looks suspiciously like the newline > character "\n". > > --Joe Atzberger > INFOhio Technical Specialist > Metropolitan Educational Council > (614) 473-8300 x22 > [EMAIL PROTECTED] > > >
