https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23766
Bug ID: 23766
Summary: Allow MARC record importer to handle large batches
without causing memory issues
Change sponsored?: ---
Product: Koha
Version: unspecified
Hardware: All
OS: All
Status: NEW
Severity: enhancement
Priority: P5 - low
Component: Architecture, internals, and plumbing
Assignee: [email protected]
Reporter: [email protected]
QA Contact: [email protected]
Right now, no matter how large a marc batch file is, the marc record stager
loads all the records into memory ( see
http://git.koha-community.org/gitweb/?p=koha.git;a=blob;f=C4/ImportBatch.pm;h=8d39ed20347132e34ab5661dedd15548df672708;hb=HEAD#l1545
).
We should break this up and process chunks of records at a time to keep the
memory footprint low for servers without enough RAM to handle this memory load.
My proposal is to modify process_batch in stage_file to loop over the records
processing X number of records at a time. X should be configurable via a
syspref.
In addition, this will require modifying C4::ImporBatch::RecordsFromMARCXMLFile
to accept an offset and a count such that it will start at the offset and
return the given count of records.
--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
