https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43028
Bug ID: 43028
Summary: Isolate one undecodable record from aborting a batch
import or revert
Initiative type: ---
Sponsorship ---
status:
Product: Koha
Version: Main
Hardware: All
OS: All
Status: NEW
Severity: enhancement
Priority: P5 - low
Component: Tools
Assignee: [email protected]
Reporter: [email protected]
QA Contact: [email protected]
Depends on: 35104
Target Milestone: ---
Bug 35104 makes Koha::Biblio::Metadata->store() throw
Koha::Exceptions::Metadata::Invalid for MARCXML it cannot save. Batch
import (C4::ImportBatch::BatchCommitRecords/_batchCommitItems), batch
revert (BatchRevertRecords), the corresponding background jobs, the
CLI tools built on them (misc/commit_file.pl,
misc/migration_tools/bulkmarcimport.pl), and
Koha::Import::Record::get_marc_record all needed updating so one bad
record doesn't take the rest of the batch down with it.
This patch:
- Catches the exception per-record in BatchCommitRecords (which
already commits per-record) and per-item in _batchCommitItems,
flagging the record/item with an 'error' status and message and
counting it via a new $num_errored return value, instead of letting
it propagate.
- Does the same for BatchRevertRecords, which runs as a single
whole-batch transaction: both the marcxml_old decode step and the
ModBiblio/ModAuthority call that follows it are isolated, so one
record that can't be restored doesn't roll back every other record
already reverted in the same batch.
- Threads the new error counts through
Koha::BackgroundJob::MARCImportCommitBatch/RevertBatch's job report,
and counts errored records towards job progress so a batch
containing them doesn't get stuck showing as still running.
- Reports the new counts in misc/commit_file.pl's CLI summary and logs
a warning line for records automatically repaired during
bulkmarcimport.pl.
- Surfaces each errored row's reason as a tooltip on its "Error"
status badge in the manage-marc-import listing.
- Makes Koha::Import::Record::get_marc_record reuse
Koha::Biblio::Metadata::repair_marcxml (bug 35104) instead of its
own weaker strip-and-retry, so staged import records get the same
recovery (including empty-datafield handling) as the interactive
cataloguing form.
Depends on bug 35104.
Test plan:
1. Apply this patch on top of bug 35104.
2. Stage a MARC file for import (Tools > Stage MARC records for
import) containing at least one deliberately malformed record
(e.g. a datafield with no subfields, or invalid control characters
that cannot be repaired) alongside normal records.
3. From Manage staged records, commit the batch. Confirm: the bad
record is flagged with an 'error' status and a tooltip explaining
why; every other record in the batch imports successfully; the job
report/CLI summary shows a "Number of records with errors" count.
4. Revert the same batch. Confirm one record whose marcxml_old cannot
be restored does not prevent the rest of the batch from reverting,
and the job report reflects the error count.
5. Run misc/migration_tools/bulkmarcimport.pl from the CLI against a
file containing a record with a recoverable fault (e.g. a
non-XML character). Confirm a WARNING line is logged for that
record and the import otherwise completes normally.
6. prove t/db_dependent/ImportBatch.t
t/db_dependent/Koha/BackgroundJob/MARCImportCommitBatch.t
Referenced Bugs:
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104
[Bug 35104] We should warn when attempting to save MARC records that contain
characters invalid in XML
--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
