[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Lucas Gass changed: What|Removed |Added Resolution|--- |FIXED Status|Pushed to oldstable |RESOLVED CC||lu...@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #23 from Matt Blenkinsop --- Nice work everyone! Pushed to oldstable for 22.11.x -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Matt Blenkinsop changed: What|Removed |Added Status|Pushed to stable|Pushed to oldstable Version(s)|23.11.00,23.05.02 |23.11.00,23.05.02,22.11.08 released in|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #22 from Martin Renvoize --- Thanks for all the hard work! Pushed to 23.05.x for the next release -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Martin Renvoize changed: What|Removed |Added Version(s)|23.11.00|23.11.00,23.05.02 released in|| Status|Pushed to master|Pushed to stable -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #21 from Tomás Cohen Arazi --- Pushed to master for 23.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Tomás Cohen Arazi changed: What|Removed |Added Status|Passed QA |Pushed to master Version(s)||23.11.00 released in|| -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #20 from Marcel de Rooy --- (In reply to Marcel de Rooy from comment #13) > Do we need additional tests in the db_dependent OAI scripts btw? Perhaps we do. Final word for RM. prove t/db_dependent/OAI/ t/db_dependent/OAI/AndSets.t .. ok t/db_dependent/OAI/Server.t ... ok t/db_dependent/OAI/Sets.t . ok All tests successful. Files=3, Tests=191, 9 wallclock secs ( 0.07 usr 0.03 sys + 6.86 cusr 0.74 csys = 7.70 CPU) Result: PASS -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #19 from Marcel de Rooy --- Created attachment 153546 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153546=edit Bug 33270: (QA follow-up) Do not change param hashref Might just be a theoretical thing now, but safer to clone. Test plan: Run t/db_dependent/Koha/Biblio/Metadata.t Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #18 from Marcel de Rooy --- Created attachment 153545 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153545=edit Bug 33270: (QA follow-up) Tidy Resolve: WARN tidiness The file is less tidy than before (bad/messy lines before: 21, now: 24) Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Marcel de Rooy changed: What|Removed |Added Attachment #153325|0 |1 is obsolete|| --- Comment #17 from Marcel de Rooy --- Created attachment 153544 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153544=edit Bug 33270: (follow-up) Handle records that fail attempt to ignore bad characters Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Marcel de Rooy changed: What|Removed |Added Attachment #152837|0 |1 is obsolete|| --- Comment #16 from Marcel de Rooy --- Created attachment 153543 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153543=edit Bug 33270: Attempt to recover from invalid metadata exception This patch uses the new record_strip_nonxml routine to attempt to display the record when it contains invalid characters Rather than silently strippg these, we warn in the logs, then add an 'about' container to the response. It is displayed nicely in the web view or sent as "INVALID_METADATA" in the xml response The 'error' codes for OAI seem to be at the request level, and the offered codes don't have a match for a bad record. Adding the about when we can recover seems the most generous response To test: Test plan, assumes using KTD default data - otherwise you need to find and import a record with encoding issues: 1 - Enable OAI-PMH system preference 2 - Browse to: http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords=marcxml/3500/0/352 3 - 500 error: Invalid data, cannot decode metadata object (biblio_metadata.id=368, biblionumber=369, format=marcxml, schema=MARC21, decoding_error=':8: parser error : PCDATA invalid Char value 31... 4 - Apply patch, restart all 5 - Reload the page 6 - It loads! 7 - Click 'Metadata' for record 369 - it succeeds! 8 - Check the logs - confirm you see a warning of the record problem 9 - Confirm 369 has an about section 10 - Check the individul 'GetRecord' response as well http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord=oai_dc=KOHA-OAI-TEST:369 Signed-off-by: Sam Lau Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Marcel de Rooy changed: What|Removed |Added Attachment #152836|0 |1 is obsolete|| --- Comment #15 from Marcel de Rooy --- Created attachment 153542 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153542=edit Bug 33270: Add record_strip_nonxml routine to Koha::Biblio::Metadata This adds a routine that can strip non xml characters form a record. It is intended for cases where we do not wish to throw an exception, but rather need to process a record to allow other work to continue To test: prove -v t/db_dependent/Koha/Biblio/Metadata.t Signed-off-by: Sam Lau Signed-off-by: Marcel de Rooy -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Marcel de Rooy changed: What|Removed |Added Status|Failed QA |Passed QA Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #14 from Nick Clemens --- Created attachment 153325 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=153325=edit Bug 33270: (follow-up) Handle records that fail attempt to ignore bad characters -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270
Marcel de Rooy changed:
What|Removed |Added
Status|Signed Off |Failed QA
--- Comment #13 from Marcel de Rooy ---
Revisiting.
This code part somehow does not look good to me in OAI::Server::ListBase:
if ($marcxml) {
$self->record(
Koha::OAI::Server::Record->new(
$repository, $marcxml, $timestamp, \@setSpecs,
%params
)
);
} else {
$self->record(
Koha::OAI::Server::DeletedRecord->new(
$timestamp, \@setSpecs, identifier =>
$repository->{koha_identifier} . ':' . $biblionumber
)
);
If there is no marcxml, you now decide that it is a DeletedRecord. But I would
assume that you should just test $deleted? It could be a normal record, but
also crashing on the strip_nonxml route.
Please clarify.
Do we need additional tests in the db_dependent OAI scripts btw?
--
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Sam Lau changed: What|Removed |Added Attachment #152650|0 |1 is obsolete|| --- Comment #12 from Sam Lau --- Created attachment 152837 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152837=edit Bug 33270: Attempt to recover from invalid metadata exception This patch uses the new record_strip_nonxml routine to attempt to display the record when it contains invalid characters Rather than silently strippg these, we warn in the logs, then add an 'about' container to the response. It is displayed nicely in the web view or sent as "INVALID_METADATA" in the xml response The 'error' codes for OAI seem to be at the request level, and the offered codes don't have a match for a bad record. Adding the about when we can recover seems the most generous response To test: Test plan, assumes using KTD default data - otherwise you need to find and import a record with encoding issues: 1 - Enable OAI-PMH system preference 2 - Browse to: http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords=marcxml/3500/0/352 3 - 500 error: Invalid data, cannot decode metadata object (biblio_metadata.id=368, biblionumber=369, format=marcxml, schema=MARC21, decoding_error=':8: parser error : PCDATA invalid Char value 31... 4 - Apply patch, restart all 5 - Reload the page 6 - It loads! 7 - Click 'Metadata' for record 369 - it succeeds! 8 - Check the logs - confirm you see a warning of the record problem 9 - Confirm 369 has an about section 10 - Check the individul 'GetRecord' response as well http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord=oai_dc=KOHA-OAI-TEST:369 Signed-off-by: Sam Lau -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Sam Lau changed: What|Removed |Added Attachment #152649|0 |1 is obsolete|| --- Comment #11 from Sam Lau --- Created attachment 152836 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152836=edit Bug 33270: Add record_strip_nonxml routine to Koha::Biblio::Metadata This adds a routine that can strip non xml characters form a record. It is intended for cases where we do not wish to throw an exception, but rather need to process a record to allow other work to continue To test: prove -v t/db_dependent/Koha/Biblio/Metadata.t Signed-off-by: Sam Lau -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Sam Lau changed: What|Removed |Added Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Nick Clemens changed: What|Removed |Added See Also||https://bugs.koha-community ||.org/bugzilla3/show_bug.cgi ||?id=34014 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #10 from Nick Clemens --- (In reply to Marcel de Rooy from comment #7) > (In reply to David Cook from comment #6) > > I recently bumped into this problem as well, although I don't try to recover > > using StripNonXmlChars()... I don't know what I think about sending out a > > slightly different version than what's stored in the database... > > Agreed. > Yes, the new code with StripNonXmlChars here in OAI is a bit weird. Either > we do such a thing in ->record or we dont? > Probably we dont. So could you only send an error and continue? > > Moving to FQA We used to do this, see bug 29697 OAI doesn't seem to have errors at the record level, only the request, and they don't match to what is happening here: https://www.openarchives.org/OAI/openarchivesprotocol.html#ErrorConditions (In reply to David Cook from comment #6) > I recently bumped into this problem as well, although I don't try to recover > using StripNonXmlChars()... I don't know what I think about sending out a > slightly different version than what's stored in the database... > > But overall we do need a fix harvests choking on 1 bad record. It is a bit odd, what we do in staff side is just show it and indiciate it may be degraded - I propose something similar here - strip them and add an 'about' section. This seems to be a more general way to send a message We should ultimately prevent these records from being added to a catalog, but data can get messy. I restrict this now to a specific exception. I think it's a better use experience to handle it and warn when we can, rather than rejext totally. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #9 from Nick Clemens --- Created attachment 152650 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152650=edit Bug 33270: Attempt to recover from invalid metadata exception This patch uses the new record_strip_nonxml routine to attempt to display the record when it contains invalid characters Rather than silently strippg these, we warn in the logs, then add an 'about' container to the response. It is displayed nicely in the web view or sent as "INVALID_METADATA" in the xml response The 'error' codes for OAI seem to be at the request level, and the offered codes don't have a match for a bad record. Adding the about when we can recover seems the most generous response To test: Test plan, assumes using KTD default data - otherwise you need to find and import a record with encoding issues: 1 - Enable OAI-PMH system preference 2 - Browse to: http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords=marcxml/3500/0/352 3 - 500 error: Invalid data, cannot decode metadata object (biblio_metadata.id=368, biblionumber=369, format=marcxml, schema=MARC21, decoding_error=':8: parser error : PCDATA invalid Char value 31... 4 - Apply patch, restart all 5 - Reload the page 6 - It loads! 7 - Click 'Metadata' for record 369 - it succeeds! 8 - Check the logs - confirm you see a warning of the record problem 9 - Confirm 369 has an about section 10 - Check the individul 'GetRecord' response as well http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord=oai_dc=KOHA-OAI-TEST:369 -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Nick Clemens changed: What|Removed |Added Attachment #152573|0 |1 is obsolete|| --- Comment #8 from Nick Clemens --- Created attachment 152649 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152649=edit Bug 33270: Add record_strip_nonxml routine to Koha::Biblio::Metadata This adds a routine that can strip non xml characters form a record. It is intended for cases where we do not wish to throw an exception, but rather need to process a record to allow other work to continue To test: prove -v t/db_dependent/Koha/Biblio/Metadata.t -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Nick Clemens changed: What|Removed |Added Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Marcel de Rooy changed: What|Removed |Added QA Contact|testo...@bugs.koha-communit |m.de.r...@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Marcel de Rooy changed: What|Removed |Added CC||m.de.r...@rijksmuseum.nl Status|Signed Off |Failed QA --- Comment #7 from Marcel de Rooy --- (In reply to David Cook from comment #6) > I recently bumped into this problem as well, although I don't try to recover > using StripNonXmlChars()... I don't know what I think about sending out a > slightly different version than what's stored in the database... Agreed. Yes, the new code with StripNonXmlChars here in OAI is a bit weird. Either we do such a thing in ->record or we dont? Probably we dont. So could you only send an error and continue? Moving to FQA -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 David Cook changed: What|Removed |Added CC||dc...@prosentient.com.au --- Comment #6 from David Cook --- I recently bumped into this problem as well, although I don't try to recover using StripNonXmlChars()... I don't know what I think about sending out a slightly different version than what's stored in the database... But overall we do need a fix harvests choking on 1 bad record. -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #5 from Nick Clemens --- (In reply to Tomás Cohen Arazi from comment #3) > Does the OAI-PMH protocol prescribe a special behavior in this cases? I suppose we should be sending an error element: http://www.openarchives.org/OAI/2.0/guidelines-repository.htm#ErrorHandling I consider this one a regression though - harvests used to ignore these, now they fail completely -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Sam Lau changed: What|Removed |Added Attachment #152550|0 |1 is obsolete|| --- Comment #4 from Sam Lau --- Created attachment 152573 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152573=edit Bug 33270: Warn errors but catch exceptions when encountering record issues in OAI-PMH This patch alters the OAI code to attempt to load a badly encoded record by removing non XML characters. OAI harvests are used to keep external catalogs or display layers in sync with Koha. We should not break this connection when there is a bad record, but should allow the harvest to complete - warn the errors, and rely on Koha warnings/errors/tools to idnetify and fix the problem on the Koha side Test plan, assumes using KTD default data - otherwise you need to find and import a record with encoding issues: 1 - Enable OAI-PMH system preference 2 - Browse to: http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords=marcxml/3500/0/352 3 - 500 error: Invalid data, cannot decode metadata object (biblio_metadata.id=368, biblionumber=369, format=marcxml, schema=MARC21, decoding_error=':8: parser error : PCDATA invalid Char value 31... 4 - Apply patch, restart all 5 - Reload the page 6 - It loads! 7 - Click 'Metadata' for record 369 - it succeeds! 8 - Check the logs - confirm you see a warning of the record problem Signed-off-by: Sam Lau -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Sam Lau changed: What|Removed |Added Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Tomás Cohen Arazi changed: What|Removed |Added CC||tomasco...@gmail.com --- Comment #3 from Tomás Cohen Arazi --- Does the OAI-PMH protocol prescribe a special behavior in this cases? -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Nick Clemens changed: What|Removed |Added Severity|normal |major -- You are receiving this mail because: You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Nick Clemens changed: What|Removed |Added Assignee|koha-b...@lists.koha-commun |n...@bywatersolutions.com |ity.org | Depends on||23846, 29697 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23846 [Bug 23846] Handle records with broken MARCXML on the bibliographic detail view https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29697 [Bug 29697] Replace GetMarcBiblio occurrences with $biblio->metadata->record -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #2 from Nick Clemens --- Created attachment 152550 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152550=edit Bug 33270: Warn errors but catch exceptions when encountering record issues in OAI-PMH This patch alters the OAI code to attempt to load a badly encoded record by removing non XML characters. OAI harvests are used to keep external catalogs or display layers in sync with Koha. We should not break this connection when there is a bad record, but should allow the harvest to complete - warn the errors, and rely on Koha warnings/errors/tools to idnetify and fix the problem on the Koha side Test plan, assumes using KTD default data - otherwise you need to find and import a record with encoding issues: 1 - Enable OAI-PMH system preference 2 - Browse to: http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords=marcxml/3500/0/352 3 - 500 error: Invalid data, cannot decode metadata object (biblio_metadata.id=368, biblionumber=369, format=marcxml, schema=MARC21, decoding_error=':8: parser error : PCDATA invalid Char value 31... 4 - Apply patch, restart all 5 - Reload the page 6 - It loads! 7 - Click 'Metadata' for record 369 - it succeeds! 8 - Check the logs - confirm you see a warning of the record problem -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 Nick Clemens changed: What|Removed |Added Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
[Koha-bugs] [Bug 33270] OAI-PMH should not die on record errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33270 --- Comment #1 from Jonathan Druart --- *** Bug 33279 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. ___ Koha-bugs mailing list Koha-bugs@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
