We are in the process of upgrading our site from 1.4.2 to 1.5.3 and we are seeing a similar error as has been described in this thread. We are seeing the error below and continued looping of the indexer trying to index the archival objects.
~~~ Indexed 1225 of 67886 archival_object records in repository 2 ( added 25 records in 6399.0ms ) ~~~ ~~~ Indexed 1250 of 67886 archival_object records in repository 2 ( added 25 records in 5488.0ms ) ~~~ E, [2017-03-16T16:53:51.425000 #779] ERROR -- : Thread-2392462: Unhandled exception! E, [2017-03-16T16:53:51.425000 #779] ERROR -- : undefined method `related_records' for nil:NilClass /opt/archivesspace-1.5.3/archivesspace/data/tmp/jetty-0.0.0.0-9089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in `top_container' /opt/archivesspace-1.5.3/archivesspace/data/tmp/jetty-0.0.0.0-9089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in `type_1' /opt/archivesspace-1.5.3/archivesspace/data/tmp/jetty-0.0.0.0-9089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in `to_hash' org/jruby/RubyHash.java:1341:in `each' org/jruby/RubyEnumerable.java:757:in `map' Does a resolution to this issue exist? How do I find what the offending set of data is causing the problem? How do I proceed? Thank you for your help. Regards, __________________________ Brent Ellingson Sr. Software Engineer BYU Library 801-422-6148 Date: Mon, 23 Jan 2017 19:54:53 +0000 From: "Majewski, Steven Dennis (sdm7g)" <[email protected]<mailto:[email protected]>> To: Archivesspace Users Group <[email protected]<mailto:[email protected]>> Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2 Message-ID: <[email protected]<mailto:[email protected]>> Content-Type: text/plain; charset="utf-8" The first two issues appear to be due to trailing spaces in some of the container barcodes. Top containers have already been created for barcodes without trailing space, and so it appears that the barcodes don?t match. On Jan 20, 2017, at 3:49 PM, Majewski, Steven Dennis (sdm7g) <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> wrote: On Jan 13, 2017, at 6:15 PM, James Bullen <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> wrote: Hi Steve, That fits the pattern I?m seeing in John?s data. Have you checked the container conversion log for any errors relating to that record? Cheers, James Just re-did migration from 1.4.2 to 1.5.1 again, and I?m inspecting the logs. Yes, archival_objects/255 is on that list. error,message,url JSONModel::ValidationException,barcode_1 -- Mismatch when mapping between barcode and barcode_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/331<http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/331> JSONModel::ValidationException,barcode_1 -- Mismatch when mapping between barcode and barcode_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/2059<http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/2059> JSONModel::ValidationException,indicator_1 -- Mismatch when mapping between indicator and indicator_1,http://UL-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/255<http://ul-sdm7g-mbp.local/resolve/readonly?uri=/repositories/3/archival_objects/255> I need to inspect the other repo logs and see if all of the ones that break are getting caught, but it looks likely. I?m trying to interpret what exactly those messages are complaining about. Do those error messages and object dumps mean anything to you off the bat? ? Steve. On Jan 14, 2017, at 8:37 AM, Majewski, Steven Dennis (sdm7g) <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> wrote: It looks like the indexer gives and error because the backend is getting an error turning the SQL model into a JSON model. For some of the archival objects that are failing to index, the significant difference is that they do not have a sub_container in their instance. Trying to access the json from the backend in 1.5.2 just yields an error message, but going back to test 1.5.1 migration: [ This is json from the same screen capture in my previous message. ] archival_object 255 ( which causes error on index ) { "lock_version": 0, "position": 17, "publish": true, "ref_id": "aspace_22a35d6a6ee59ac454ca8f35232b0b40", "title": "Buttons and uniform parts, souvenirs, and medals", "display_string": "Buttons and uniform parts, souvenirs, and medals, undated", "restrictions_apply": false, "created_by": "admin", "last_modified_by": "admin", "create_time": "2015-08-11T21:43:55Z", "system_mtime": "2017-01-06T22:25:38Z", "user_mtime": "2015-08-11T21:43:55Z", "suppressed": false, "level": "file", "jsonmodel_type": "archival_object", "external_ids": [], "subjects": [], "linked_events": [], "extents": [], "dates": [ { "lock_version": 0, "expression": "undated", "created_by": "admin", "last_modified_by": "admin", "create_time": "2015-08-11T21:43:55Z", "system_mtime": "2015-08-11T21:43:55Z", "user_mtime": "2015-08-11T21:43:55Z", "date_type": "inclusive", "label": "creation", "jsonmodel_type": "date" } ], "external_documents": [], "rights_statements": [], "linked_agents": [], "instances": [ { "lock_version": 0, "created_by": "admin", "last_modified_by": "admin", "create_time": "2015-08-11T21:43:55Z", "system_mtime": "2015-08-11T21:43:55Z", "user_mtime": "2015-08-11T21:43:55Z", "instance_type": "Realia", "jsonmodel_type": "instance", "is_representative": false, "container": { "lock_version": 0, "indicator_1": "Artifacts 1", "barcode_1": "X030898965", "created_by": "admin", "last_modified_by": "admin", "create_time": "2015-08-11T21:43:55Z", "system_mtime": "2016-11-22T16:58:54Z", "user_mtime": "2015-08-11T21:43:55Z", "type_1": "box", "jsonmodel_type": "container", "container_locations": [] } } ], "notes": [], "uri": "/repositories/3/archival_objects/255", "repository": { "ref": "/repositories/3" }, "resource": { "ref": "/repositories/3/resources/11" }, "has_unpublished_ancestor": false } archival_object 256: ( indexes successfully, once 255 is deleted. ) { "lock_version": 0, "position": 18, "publish": false, "ref_id": "aspace_15da7f2a9b0184a65b1280ae05a7598a", "title": "Buttons and uniform parts, souvenirs, and medals", "display_string": "Buttons and uniform parts, souvenirs, and medals, undated", "restrictions_apply": false, "created_by": "admin", "last_modified_by": "admin", "create_time": "2015-08-11T21:43:55Z", "system_mtime": "2017-01-06T22:25:38Z", "user_mtime": "2015-08-11T21:43:55Z", "suppressed": false, "level": "file", "jsonmodel_type": "archival_object", "external_ids": [], "subjects": [], "linked_events": [], "extents": [], "dates": [ { "lock_version": 0, "expression": "undated", "created_by": "admin", "last_modified_by": "admin", "create_time": "2015-08-11T21:43:55Z", "system_mtime": "2015-08-11T21:43:55Z", "user_mtime": "2015-08-11T21:43:55Z", "date_type": "inclusive", "label": "creation", "jsonmodel_type": "date" } ], "external_documents": [], "rights_statements": [], "linked_agents": [], "instances": [ { "lock_version": 0, "created_by": "admin", "last_modified_by": "admin", "create_time": "2015-08-11T21:43:55Z", "system_mtime": "2015-08-11T21:43:55Z", "user_mtime": "2015-08-11T21:43:55Z", "instance_type": "Realia", "jsonmodel_type": "instance", "is_representative": false, "sub_container": { "lock_version": 0, "created_by": "admin", "last_modified_by": "admin", "create_time": "2017-01-06T22:28:29Z", "system_mtime": "2017-01-06T22:28:29Z", "user_mtime": "2017-01-06T22:28:29Z", "jsonmodel_type": "sub_container", "top_container": { "ref": "/repositories/3/top_containers/1573" } }, "container": { "type_1": "box", "indicator_1": "Artifacts 2", "barcode_1": "X030866782", "container_locations": [], "lock_version": 1 } } ], "notes": [], "uri": "/repositories/3/archival_objects/256", "repository": { "ref": "/repositories/3" }, "resource": { "ref": "/repositories/3/resources/11" }, "has_unpublished_ancestor": false } But as I noted previously, I don?t see the significant difference in the data in 1.4.2 that was migrated that would account for one failing and the other succeeding. ( It appears that there are other records causing errors that don?t necessarily fit this same pattern. ) But the fact that this info is missing in the 1.5.1 migration does suggest it?s not a 1.5.2 only bug, but a problem with the container migration in earlier versions that is just made more visible by some 1.5.2 changes. ? Steve. On Jan 12, 2017, at 11:39 PM, James Bullen <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> wrote: Hi, It looks like the problem John is seeing with the indexer failing after upgrading to 1.5.2 is caused by an issue with a record following the container conversion. I?m following up with John off-list on the specifics of his problem. Hopefully we?ll be able to confirm this when we get to a resolution. The container management upgrade has been the source of quite a few bugs and it is on our radar for review. Cheers, James On Jan 13, 2017, at 1:30 AM, Hambleton, John S <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> wrote: Yes, thank you, I sent my data to you off-list. John H NMU From: [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> [mailto:[email protected]] On Behalf Of James Bullen Sent: Wednesday, January 11, 2017 6:56 PM To: Archivesspace Users Group <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2 Hi John and Steve, Can either of you send some sample data that surfaces this bug? Cheers, James On Jan 12, 2017, at 8:17 AM, Majewski, Steven Dennis (sdm7g) <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> wrote: After seeing this message, I tried 1.5.2 on my Mac (10.11.6) laptop and I?m also seeing the same undefined method `related_records' for nil:NilClass error, whether or not I do an intervening ( and successful, BTW ) update to 1.5.1. Reindexing never seems to progress and so browse resources never shows any records. ? Steve Majewski On Jan 11, 2017, at 1:58 PM, Hambleton, John S <[email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>>> wrote: Folks, I am testing archivesspace 1.5.2 on CentOS 6, using a copy of our 1.4.2 production database. Wondering why it seems like the indexer is taking so long to finish, I am seeing this in the archivesspace.out log, which makes me think the indexer is caught in an endless loop: Keep seeing the SAME indexer messages over and over again: ~~~ Indexed 1475 of 1941 accession records in repository 3 ( added 25 records in 8229.0ms ) ~~~ ~~~ Indexed 1500 of 1941 accession records in repository 3 ( added 25 records in 5251.0ms ) ~~~ And then, E, [2017-01-11T13:41:10.740000 #32733] ERROR -- : Thread-6445852: Unhandled exception! E, [2017-01-11T13:41:10.741000 #32733] ERROR -- : undefined method `related_records' for nil:NilClass /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in `top_container' /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in `type_1' /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in `to_hash' org/jruby/RubyHash.java:1341:in `each' And then seems to start indexing the same data over again: ~~~ Indexed 50 of 1941 accession records in repository 3 ( added 25 records in 11153.0ms ) ~~~ ~~~ Indexed 75 of 1941 accession records in repository 3 ( added 25 records in 10051.0ms ) ~~~ ~~~ Indexed 100 of 1941 accession records in repository 3 ( added 25 records in 9558.0ms ) ~~~ Kind of looks like the indexer errors on ?something? then, tries to reindex the same accession records over and over again. I had followed the advice here:https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion before starting archivesspace, that is, ?delete your Solr index files to start with a fresh index?. Any help is appreciated. Thanks, John H NMU _______________________________________________ Archivesspace_Users_Group mailing list [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group _______________________________________________ Archivesspace_Users_Group mailing list [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group !DSPAM:587793334611909212977! _______________________________________________ Archivesspace_Users_Group mailing list [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group !DSPAM:587793334611909212977! _______________________________________________ Archivesspace_Users_Group mailing list [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group !DSPAM:587948af109451970717657! _______________________________________________ Archivesspace_Users_Group mailing list [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group !DSPAM:587948af109451970717657! _______________________________________________ Archivesspace_Users_Group mailing list [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group <7ebc58861f8c38997d24d1316e9276ec>_______________________________________________ Archivesspace_Users_Group mailing list [email protected]<mailto:[email protected]<mailto:[email protected]%3cmailto:[email protected]>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________ Archivesspace_Users_Group mailing list [email protected] http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
