Hi Steve, That fits the pattern I’m seeing in John’s data. Have you checked the container conversion log for any errors relating to that record?
Cheers, James > On Jan 14, 2017, at 8:37 AM, Majewski, Steven Dennis (sdm7g) > <[email protected]> wrote: > > > It looks like the indexer gives and error because the backend is getting an > error turning the SQL model into a JSON model. > > For some of the archival objects that are failing to index, the significant > difference is that they do not have a sub_container in their instance. Trying > to access the json from the backend in 1.5.2 just yields an error message, > but going back to test 1.5.1 migration: > > [ This is json from the same screen capture in my previous message. ] > > archival_object 255 ( which causes error on index ) > > { > "lock_version": 0, > "position": 17, > "publish": true, > "ref_id": "aspace_22a35d6a6ee59ac454ca8f35232b0b40", > "title": "Buttons and uniform parts, souvenirs, and medals", > "display_string": "Buttons and uniform parts, souvenirs, and medals, > undated", > "restrictions_apply": false, > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2015-08-11T21:43:55Z", > "system_mtime": "2017-01-06T22:25:38Z", > "user_mtime": "2015-08-11T21:43:55Z", > "suppressed": false, > "level": "file", > "jsonmodel_type": "archival_object", > "external_ids": [], > "subjects": [], > "linked_events": [], > "extents": [], > "dates": [ > { > "lock_version": 0, > "expression": "undated", > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2015-08-11T21:43:55Z", > "system_mtime": "2015-08-11T21:43:55Z", > "user_mtime": "2015-08-11T21:43:55Z", > "date_type": "inclusive", > "label": "creation", > "jsonmodel_type": "date" > } > ], > "external_documents": [], > "rights_statements": [], > "linked_agents": [], > "instances": [ > { > "lock_version": 0, > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2015-08-11T21:43:55Z", > "system_mtime": "2015-08-11T21:43:55Z", > "user_mtime": "2015-08-11T21:43:55Z", > "instance_type": "Realia", > "jsonmodel_type": "instance", > "is_representative": false, > "container": { > "lock_version": 0, > "indicator_1": "Artifacts 1", > "barcode_1": "X030898965", > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2015-08-11T21:43:55Z", > "system_mtime": "2016-11-22T16:58:54Z", > "user_mtime": "2015-08-11T21:43:55Z", > "type_1": "box", > "jsonmodel_type": "container", > "container_locations": [] > } > } > ], > "notes": [], > "uri": "/repositories/3/archival_objects/255", > "repository": { > "ref": "/repositories/3" > }, > "resource": { > "ref": "/repositories/3/resources/11" > }, > "has_unpublished_ancestor": false > } > > archival_object 256: ( indexes successfully, once 255 is deleted. ) > > { > "lock_version": 0, > "position": 18, > "publish": false, > "ref_id": "aspace_15da7f2a9b0184a65b1280ae05a7598a", > "title": "Buttons and uniform parts, souvenirs, and medals", > "display_string": "Buttons and uniform parts, souvenirs, and medals, > undated", > "restrictions_apply": false, > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2015-08-11T21:43:55Z", > "system_mtime": "2017-01-06T22:25:38Z", > "user_mtime": "2015-08-11T21:43:55Z", > "suppressed": false, > "level": "file", > "jsonmodel_type": "archival_object", > "external_ids": [], > "subjects": [], > "linked_events": [], > "extents": [], > "dates": [ > { > "lock_version": 0, > "expression": "undated", > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2015-08-11T21:43:55Z", > "system_mtime": "2015-08-11T21:43:55Z", > "user_mtime": "2015-08-11T21:43:55Z", > "date_type": "inclusive", > "label": "creation", > "jsonmodel_type": "date" > } > ], > "external_documents": [], > "rights_statements": [], > "linked_agents": [], > "instances": [ > { > "lock_version": 0, > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2015-08-11T21:43:55Z", > "system_mtime": "2015-08-11T21:43:55Z", > "user_mtime": "2015-08-11T21:43:55Z", > "instance_type": "Realia", > "jsonmodel_type": "instance", > "is_representative": false, > "sub_container": { > "lock_version": 0, > "created_by": "admin", > "last_modified_by": "admin", > "create_time": "2017-01-06T22:28:29Z", > "system_mtime": "2017-01-06T22:28:29Z", > "user_mtime": "2017-01-06T22:28:29Z", > "jsonmodel_type": "sub_container", > "top_container": { > "ref": "/repositories/3/top_containers/1573" > } > }, > "container": { > "type_1": "box", > "indicator_1": "Artifacts 2", > "barcode_1": "X030866782", > "container_locations": [], > "lock_version": 1 > } > } > ], > "notes": [], > "uri": "/repositories/3/archival_objects/256", > "repository": { > "ref": "/repositories/3" > }, > "resource": { > "ref": "/repositories/3/resources/11" > }, > "has_unpublished_ancestor": false > } > > > > But as I noted previously, I don’t see the significant difference in the data > in 1.4.2 that was migrated that would account for one failing and the other > succeeding. > > > ( It appears that there are other records causing errors that don’t > necessarily fit this same pattern. ) > > But the fact that this info is missing in the 1.5.1 migration does suggest > it’s not a 1.5.2 only bug, but a problem with the container migration in > earlier versions that is just made more visible by some 1.5.2 changes. > > > — Steve. > > > >> On Jan 12, 2017, at 11:39 PM, James Bullen <[email protected] >> <mailto:[email protected]>> wrote: >> >> >> Hi, >> >> It looks like the problem John is seeing with the indexer failing after >> upgrading to 1.5.2 is caused by an issue with a record following the >> container conversion. I’m following up with John off-list on the specifics >> of his problem. Hopefully we’ll be able to confirm this when we get to a >> resolution. >> >> The container management upgrade has been the source of quite a few bugs and >> it is on our radar for review. >> >> >> Cheers, >> James >> >> >> >>> On Jan 13, 2017, at 1:30 AM, Hambleton, John S <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Yes, thank you, >>> I sent my data to you off-list. >>> >>> John H >>> NMU >>> >>> From: [email protected] >>> <mailto:[email protected]> >>> [mailto:[email protected] >>> <mailto:[email protected]>] On Behalf >>> Of James Bullen >>> Sent: Wednesday, January 11, 2017 6:56 PM >>> To: Archivesspace Users Group >>> <[email protected] >>> <mailto:[email protected]>> >>> Subject: Re: [Archivesspace_Users_Group] Indexer erroring in 1.5.2 >>> >>> >>> Hi John and Steve, >>> >>> Can either of you send some sample data that surfaces this bug? >>> >>> >>> Cheers, >>> James >>> >>> >>> On Jan 12, 2017, at 8:17 AM, Majewski, Steven Dennis (sdm7g) >>> <[email protected] <mailto:[email protected]>> wrote: >>> >>> >>> After seeing this message, I tried 1.5.2 on my Mac (10.11.6) laptop and I’m >>> also seeing the same >>> undefined method `related_records' for nil:NilClass >>> error, whether or not I do an intervening ( and successful, BTW ) update to >>> 1.5.1. >>> Reindexing never seems to progress and so browse resources never shows any >>> records. >>> >>> >>> — Steve Majewski >>> >>> >>> >>> On Jan 11, 2017, at 1:58 PM, Hambleton, John S <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Folks, >>> >>> I am testing archivesspace 1.5.2 on CentOS 6, using a copy of our 1.4.2 >>> production database. >>> Wondering why it seems like the indexer is taking so long to finish, I am >>> seeing this in >>> the archivesspace.out log, which makes me think the indexer is caught in an >>> endless loop: >>> >>> Keep seeing the SAME indexer messages over and over again: >>> ~~~ Indexed 1475 of 1941 accession records in repository 3 ( added 25 >>> records in 8229.0ms ) ~~~ >>> ~~~ Indexed 1500 of 1941 accession records in repository 3 ( added 25 >>> records in 5251.0ms ) ~~~ >>> >>> And then, >>> >>> E, [2017-01-11T13:41:10.740000 #32733] ERROR -- : Thread-6445852: Unhandled >>> exception! >>> E, [2017-01-11T13:41:10.741000 #32733] ERROR -- : >>> undefined method `related_records' for nil:NilClass >>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:64:in >>> `top_container' >>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:24:in >>> `type_1' >>> /usr/local/archivesspace/data/tmp/jetty-0.0.0.0-8089-backend.war-_-any-/webapp/WEB-INF/app/lib/subcontainer_to_aspace_json_mapper.rb:14:in >>> `to_hash' >>> org/jruby/RubyHash.java:1341:in `each' >>> >>> And then seems to start indexing the same data over again: >>> ~~~ Indexed 50 of 1941 accession records in repository 3 ( added 25 records >>> in 11153.0ms ) ~~~ >>> ~~~ Indexed 75 of 1941 accession records in repository 3 ( added 25 records >>> in 10051.0ms ) ~~~ >>> ~~~ Indexed 100 of 1941 accession records in repository 3 ( added 25 >>> records in 9558.0ms ) ~~~ >>> >>> Kind of looks like the indexer errors on “something” then, tries to reindex >>> the same accession >>> records over and over again. I had followed the advice >>> here:https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion >>> >>> <https://github.com/archivesspace/archivesspace/blob/master/UPGRADING_1.5.0.md#conversion> >>> before starting archivesspace, that is, “delete your Solr index files to >>> start with a fresh index”. >>> >>> Any help is appreciated. >>> Thanks, >>> John H >>> NMU >>> >>> _______________________________________________ >>> Archivesspace_Users_Group mailing list >>> [email protected] >>> <mailto:[email protected]> >>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group >>> <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group> >>> >>> _______________________________________________ >>> Archivesspace_Users_Group mailing list >>> [email protected] >>> <mailto:[email protected]> >>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group >>> <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group> >>> >>> >>> >>> !DSPAM:587793334611909212977! >>> _______________________________________________ >>> Archivesspace_Users_Group mailing list >>> [email protected] >>> <mailto:[email protected]> >>> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group >>> <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group> >>> >>> >>> !DSPAM:587793334611909212977! >> >> _______________________________________________ >> Archivesspace_Users_Group mailing list >> [email protected] >> <mailto:[email protected]> >> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > > !DSPAM:587948af109451970717657! > _______________________________________________ > Archivesspace_Users_Group mailing list > [email protected] > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > > > !DSPAM:587948af109451970717657!
_______________________________________________ Archivesspace_Users_Group mailing list [email protected] http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
