> -----Original Message-----
> From: Shelley Doljack [mailto:sdolj...@stanford.edu]
> Sent: 31 July 2012 20:18
> The problem was I wasn't telling perl to output UTF-8. Now that I added
> binmode(FILE, ':utf8') to my script, the problem is fixed. However, it sounds
> like once I set binmode to UTF-8 everything will be interpreted as such, even
> when the record is in MARC-8. Is that right? So this means that I can only use
> my script with a file of records where all of them are encoded in UTF-8. If I
> want to run the script against a file with all MARC-8 encoding, then I'd need
> to remove the binmode line.

It depends how much manipulation of the records you are doing in the script.  
One approach is to use

binmode(FILE, ':raw');

for both input and output.  Perl will then keep the bytes of the records 
exactly as they are.  You won't be able to test  for exotic characters so 
easily, and amending field content would be inadvisable, but if all you are 
doing is something like reading in the records and filtering out any that have 
no 245 field, or something fairly basic like that, this could be the best 

The MARC::Record module does not seem to care how the records are encoded.  
It's only once you start altering field content, testing field content, or 
adding fields that the character set being used becomes an issue.  Removing 
fields would be fine too.

MARC-8 can be very complex, particularly if other code tables like CJK are 
invoked, or even just Greek or Cyrillic.  If you were manipulating field 
content in that kind of way they converting everything to UTF-8 would make 
things very much easier.


Matthew Phillips
Electronic Systems Librarian, Durham University
Durham University Library, Stockton Road, Durham, DH1 3LY
+44 (0)191 334 2941

Reply via email to