We've seen similar behavior with Fedora 3.2.1 and 3.3 when ingesting managed datastreams with checksums using the Fedora client ingest script at /client/bin/fedora-ingest.sh. We're temporarily working around this issue by changing these datastreams to type 'E', for which we have not observed this problem.
DEBUG 2010-04-01 12:57:30.086 [http-80-Processor23] (FOXMLDODeserializer) instantiate datastream: dsid = ALTOchecksumType = MD5checksum = 44a41b4c4daab42d8d600796b205a6a2 DEBUG 2010-04-01 12:57:30.086 [http-80-Processor23] (FOXMLDODeserializer) New Object: checking supplied checksum DEBUG 2010-04-01 12:57:30.086 [http-80-Processor23] (Datastream) checksumType is MD5 DEBUG 2010-04-01 12:57:30.086 [http-80-Processor23] (Datastream) Classname = fedora.server.storage.types.DatastreamManagedContent DEBUG 2010-04-01 12:57:30.086 [http-80-Processor23] (Datastream) location = http://localhost/foo/bar/ALTO00001.xml DEBUG 2010-04-01 12:57:30.086 [http-80-Processor23] (DefaultExternalContentManager) in getExternalContent(), url=http://localhost/foo/bar/ALTO00001.xml DEBUG 2010-04-01 12:57:30.095 [http-80-Processor23] (DefaultExternalContentManager) ************************* backendUsername: backendPassword: backendSSL: false DEBUG 2010-04-01 12:57:30.095 [http-80-Processor23] (DefaultExternalContentManager) ************************* doAuthnGetURL: http://localhost/foo/bar/ALTO00001.xml DEBUG 2010-04-01 12:57:30.095 [http-80-Processor23] (DefaultExternalContentManager) DefaultExternalContentManager.get(http://localhost/foo/bar/ALTO00001.xml) DEBUG 2010-04-01 12:57:30.137 [http-80-Processor23] (DefaultAuthorization) Entered enforceUpload [...] DEBUG 2010-04-01 12:57:30.137 [http-80-Processor23] (DefaultAuthorization) Exiting enforceUpload WARN 2010-04-01 12:57:30.137 [http-80-Processor23] (Datastream) IOException reading datastream to generate checksum DEBUG 2010-04-01 12:57:30.137 [http-80-Processor23] (Datastream) Checksum = ExceptionReadingStream DEBUG 2010-04-01 12:57:30.137 [http-80-Processor23] (FOXMLDODeserializer) checksum = ExceptionReadingStream DEBUG 2010-04-01 12:57:30.137 [http-80-Processor23] (DefaultDOManager) Finally, removing temp file INFO 2010-04-01 12:57:30.137 [http-80-Processor23] (DefaultManagement) Completed ingest(objectXML, format: info:fedora/fedora-system:FOXML-1.1, encoding: UTF-8, newPid: true, logMessage: Ingested from local file /foo/bar/FOXML-9516877-pg-1.xml) DEBUG 2010-04-01 12:57:30.137 [http-80-Processor23] (DefaultManagement) Exiting ingest DEBUG 2010-04-01 12:57:30.137 [http-80-Processor23] (DefaultManagement) Memory: 557801880 bytes free of 652607488 available. ERROR 2010-04-01 12:57:30.138 [http-80-Processor23] (FedoraAPIMBindingSOAPHTTPImpl) Error ingesting fedora.server.errors.ObjectIntegrityException: FOXML IO stream was bad : Checksum Mismatch: ExceptionReadingStream at fedora.server.storage.translation.FOXMLDODeserializer.deserialize(FOXMLDODeserializer.java:252) at fedora.server.storage.translation.DOTranslatorImpl.deserialize(DOTranslatorImpl.java:75) at fedora.server.storage.translation.DOTranslatorModule.deserialize(DOTranslatorModule.java:126) at fedora.server.storage.DefaultDOManager.getIngestWriter(DefaultDOManager.java:799) at fedora.server.management.DefaultManagement.ingest(DefaultManagement.java:136) [...] - Doron Shalvi -----Original Message----- Date: Wed, 12 May 2010 14:49:35 +0200 From: Kai Strnad <[email protected]> Subject: Re: [Fedora-commons-users] problems with checksum and managed datastreams To: Chris Beer <[email protected]> Cc: "[email protected]" <[email protected]> Message-ID: <[email protected]> Content-Type: text/plain; charset=ISO-8859-1 Hi Chris, Thanks for reporting this. I can confirm your observation, it's a bug. I've created a tracker item for it here: http://fedora-commons.org/jira/browse/FCREPO-696. - Kai > Hi all, > > I'm using Fedora 3.3 and I'm having a little bit of trouble with using > checksummed managed datastreams through the REST client directly or the Flex > client, and before digging further, I'd like to find out if any one else has > run into anything similar: > > Using the REST API: > > INFO 2010-05-11 19:03:41.341 [http-8080-2] (DefaultManagement) Completed > addDatastream(pid: file:5, dsID: File, altIDs: , dsLabel: fo-to-solr.xsl, > versionable: true, MIMEType: application/xml, formatURI: null, dsLocation: > file:/opt/scripts/fo-to-solr.xsl, controlGroup: M, dsState: A, checksumType: > MD5, checksum: 2903fd799d56a1fa15cd614a346cc71a, logMessage: null) > ERROR 2010-05-11 19:03:41.341 [http-8080-2] (DatastreamResource) Unexpected > error fulfilling REST API request > fedora.server.errors.ValidationException: Checksum Mismatch: > ExceptionReadingStream > > --- > > Using the Fedora Admin Flex client: > INFO 2010-05-11 19:07:00.070 [http-8080-1] (DefaultDOManager) Committing > modification of file:5 > INFO 2010-05-11 19:07:00.072 [http-8080-1] (DefaultDOManager) Getting managed > datastream from internal uploaded location: uploaded://6 > INFO 2010-05-11 19:07:00.101 [http-8080-1] (DefaultDOManager) Replaced > managed datastream location with internal id: file:5+File+File.0 > ERROR 2010-05-11 19:07:00.117 [http-8080-1] (DefaultExternalContentManager) > URI is not hierarchical > java.lang.IllegalArgumentException: URI is not hierarchical > [...] > WARN 2010-05-11 19:07:00.129 [http-8080-1] (Datastream) IOException reading > datastream to generate checksum > > > Thanks, > Chris Beer > Web Developer > WGBH Interactive ------------------------------------------------------------------------------ _______________________________________________ Fedora-commons-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
