Hi, all. NOTE: This is not https://bugs.launchpad.net/evergreen/+bug/1671845. It may be related or have a similar cause, but the experience/symptoms are completely different.
At this point, consider this a head's up, as well as a problem description that I don't yet think I have enough information to file as a bug report. It is also a request for anyone who wants to double check this report and to help with debugging. I've noticed some bizarre behavior with DBI, MARC::Record, and writing to a file with Perl version 5.20 and 5.22. (These versions of Perl ship with Debian 8 Jessie and Ubuntu 16.04 Xenial Xerus, respectively.) I have a script (https://github.com/Dyrcona/boopsie) that I use to make a weekly extract of records to send to Boopsie, Inc. on behalf of our member libraries that use their app. What I have seen is that when run with the aforementioned versions of Perl, the program consumes all of the RAM on the server and gets killed by OOM killer. No output ever reaches the file. This suggests to me that the problem occurs in the main loop with extract and converting the MARCXML from the database, though it could be the Perl output buffer run amok. The main loop of my program is similar, though less complicated, than that of marc_export. I tried marc_export to see if it would have the same problem. When extracting my whole database, it does: marc_export -a -e UTF-8 > all.mrc It also crashes if fed the output of an equivalent psql query to extract all of the record ids, or if a file of all record ids is piped into marc_export. It makes no difference if the output format is USMARC or MARCXML. I can split this up into batches of 50,000 or so records (quite possibly more) and all is well. I figured this out by dumping records for a branch with around 51,000 items and that worked. My whole database has just over 2.7 million, non-deleted bib records. This worked on Perl version 5.14 on Debian 7 Wheezy and on Ubuntu 14.04 Trusty Tahr. I hope to run marc_export with the Perl debugger to figure out the exact cause. Until this is fixed, I'm using a work around in my scripts of dumping MARCXML batches and converting them to USMARC and putting them into 1 file with yaz-marcdump. This seems to work in light of the Lp bug mentioned in the NOTE. Any and all information, contradictory or otherwise, from those using Debian 8 or Ubuntu 16.04 is most welcome. Jason