On Fri, 2 May 2014 16:50:25 +0000, Kyle Tomita <[email protected]> wrote: > I recently ran the auth to bib linking with the use of the batcher > scripts from MVLC, > http://blog.mvlcstaff.org/2012/09/howto-batch-authority-control.html. > > It took about 8 days and 8 hours to complete. > 800K bib records and 560K auth records > Ran batches of 5000 bibs and ran 3 batches at a time. > > I am wondering if others have used this batching process and how my > times match up.
Last time I tried running authority_control_fields.pl, it took about 30 minutes to process a set of 1,554 bib records on a test server, i.e. about 1.2s per record. Since it was just a small test set, I didn't try running multiple processes in parallel. I didn't notice particularly high load during the test, so I suspect disk I/O was the limiting factor; I'd expect better results on a db server using SSDs, but haven't tested that so far. -- Jeff Davis Lead Evergreen Specialist BC Libraries Cooperative
