I'm pretty sure it is 3.4.0 (from files on the server it looks like an August 2010 build. The server is in a totally isolated network with nothing with GUI support that can hit the admin tools).
Tomcat is the version bundled with the Fedora installer. Would you like me to be sure I'm running at the latest version and try the test scripts again before you go forward? Scott On 05/17/2011 12:45 PM, Scott Prater wrote: > Thanks, Scott. I'll try to reproduce the problem in my environment, > Fedora 3.4.2. > > Can you tell me what version of Fedora and Tomcat (or other webapp > server) you're using? > > -- Scott > > On 05/17/2011 11:08 AM, Scott Hammel wrote: >> Hey, Scott, >> >> Thanks for responding. I'm more a C/C++ programmer and not a Java >> programmer (though I sometimes play one on the Internet), so I'm just >> guessing on the array bounds -- feels like something incrementing an int >> into the sign bit, though I'd think Java would throw some array bounds >> exception before that happened. Figured I'd do a little math later maybe >> to test my hypothesis. >> >> Recall, this was all in a 32-bit environment. I really hope it is a >> non-issue and something I'm doing in the end. Note disseminating the >> datastream content directly appears to work OK, which confuses me a >> little, though I haven't looked to see if the code for that does things >> differently. >> >> Anyway, here's a series of commands (extracted from my test scripts) >> that should reproduce the problem: >> >> mkdir /usr/fedora/tomcat/webapps/ROOT/ingestpool >> mkdir /tmp/fedrun >> dir=/tmp/fedrun >> pid=test:pid01 >> >> dd if=/dev/urandom >> of=/usr/fedora/tomcat/webapps/ROOT/ingestpool/sample.bin bs=1M count=400 >> >> ./makefoxml $pid http://localhost:8080/ingestpool/sample.bin> >> $dir/sample.xml >> >> /usr/fedora/client/bin/fedora-ingest.sh f $dir/sample.xml >> info:fedora/fedora-system:FOXML-1.1 localhost:8080 fedoraAdmin<insert >> pwd here> http >> >> wget -O $dir/export.xml --auth-no-challenge --http-user=fedoraAdmin >> --http-password=<insert pwd here> >> http://localhost:8080/fedora/objects/$pid/export?context=archive >> >> Note: I use the REST call via a wget rather than the provided export >> client scripts because it looks to me from the Java heap explosion that >> the export scripts must end up doing the export via the SOAP API. >> -- >> The content of makefoxml: >> >> #!/bin/bash >> >> #usage: makefoxml<pid> <refurl> >> #escape slashes off the URL >> RF=${2//\//\\/} >> #if you need to escape ampersands as well, uncomment this: >> #RF=${RF//'&'/'\&'} >> >> # make substitutions .... >> sed ' >> s/PID=""/PID="'"$1"'"/ >> s/rdf:about=""/rdf:about="info:fedora\/'"$1"'"/ >> s/dc:identifier>/dc:identifier>'"$1"'/ >> s/REF=""/REF="'"${RF}"'"/ >> '< "foxml_tpl.xml" >> >> -- >> The content of foxml_tmp.xml (the sed script above does the edits noted >> in the xml comments in this template): >> >> <?xml version="1.0" encoding="UTF-8"?> >> <!-- following element: set the PID attribute --> >> <foxml:digitalObject VERSION="1.1" PID="" >> xmlns:foxml="info:fedora/fedora-system:def/foxml#" >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> xsi:schemaLocation="info:fedora/fedora-system:def/foxml# >> http://www.fedora.info/definitions/1/0/foxml1-1.xsd";> >> >> <foxml:objectProperties> >> <foxml:property NAME="info:fedora/fedora-system:def/model#state" VALUE="A"/> >> <foxml:property NAME="info:fedora/fedora-system:def/model#label" VALUE=""/> >> <foxml:property NAME="info:fedora/fedora-system:def/model#ownerId" >> VALUE="fedoraAdmin"/> >> </foxml:objectProperties> >> >> <foxml:datastream CONTROL_GROUP="X" ID="RELS-EXT"> >> <foxml:datastreamVersion >> FORMAT_URI="info:fedora/fedora-system:FedoraRELSExt-1.0" >> ID="RELS-EXT.0" LABEL="RDF Statements about this Object" >> MIMETYPE="application/rdf+xml"> >> <foxml:xmlContent> >> <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/"; >> xmlns:fedora="info:fedora/fedora-system:def/relations-external#" >> xmlns:fedora-model="info:fedora/fedora-system:def/model#" >> xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"; >> xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; >> xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#";> >> <!-- following element: put the PID as the value for the rdf:about >> attribute --> >> <rdf:description rdf:about=""> >> </rdf:description> >> </rdf:RDF> >> </foxml:xmlContent> >> </foxml:datastreamVersion> >> </foxml:datastream> >> >> <foxml:datastream CONTROL_GROUP="X" ID="DC" STATE="A" VERSIONABLE="true"> >> <foxml:datastreamVersion ID="DC.0" LABEL="Dublin Core Record" >> MIMETYPE="text/xml"> >> <foxml:xmlContent> >> <oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/"; >> xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"; >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ >> http://www.openarchives.org/OAI/2.0/oai_dc.xsd";> >> <dc:title></dc:title> >> <dc:creator>Test Program</dc:creator> >> <dc:description>A test object</dc:description> >> <!-- following element: put the PID between the tags --> >> <dc:identifier></dc:identifier> >> </oai_dc:dc> >> </foxml:xmlContent> >> </foxml:datastreamVersion> >> </foxml:datastream> >> >> <foxml:datastream CONTROL_GROUP="M" ID="Content" STATE="A"> >> <foxml:datastreamVersion ID="Content.0" LABEL="This is the object >> content" MIMETYPE=" application/octet-stream"> >> <!-- following element: put the URL to the content file as the value for >> the REF attribute --> >> <!-- must be an http URL, e.g., >> http://localhost:8080/ingestpool/foxmldoc.xml --> >> <!-- I just create a directory "ingestpool" under >> /usr/fedora/tomcat/webapps/ROOT and put the files there --> >> <foxml:contentLocation REF="" TYPE="URL" /> >> </foxml:datastreamVersion> >> </foxml:datastream> >> >> >> </foxml:digitalObject> >> >> >> >> On 05/17/2011 10:00 AM, Scott Prater wrote: >>> Scott, >>> >>> Can you come up with a test case that confirms this limitation? If you >>> can provide one, I'll open up a JIRA ticket for the issue. >>> >>> thanks, >>> >>> -- Scott >>> >>> On 05/16/2011 10:45 AM, Scott Hammel wrote: >>>> Oh, I think I see: last line of the serializer's serialize function does >>>> this: >>>> bytes.toByteArray() >>>> where bytes is a ByteArrayOutputStream >>>> >>>> I *think* the max size of an array index in Java (32-bit) is >>>> 2,147,483,647 (i.e., 2^31 - 1, max value of a java int). So, this >>>> function will throw an exception if a datastream "archive" export is> >>>> ~2 GB. >>>> >>>> scott >>>> >>>> On 05/16/2011 11:00 AM, Scott Hammel wrote: >>>>> Hi, >>>>> >>>>> Running some export tests using Fedora's REST export API, I get a >>>>> negative array index Java exception when doing an "archive" export of an >>>>> object at around 400 MB (>320 MB,< 450 MB). >>>>> >>>>> Fedora is version 3.4 something; running on 32-bit CentOS 5.5, Sun Java >>>>> 1.6, 21 >>>>> >>>>> Is it just me or has anyone else seen something like that? >>>>> >>>>> Thanks, >>>>> Scott >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> Achieve unprecedented app performance and reliability >>>>> What every C/C++ and Fortran developer should know. >>>>> Learn how Intel has extended the reach of its next-generation tools >>>>> to help boost performance applications - inlcuding clusters. >>>>> http://p.sf.net/sfu/intel-dev2devmay >>>>> _______________________________________________ >>>>> Fedora-commons-users mailing list >>>>> Fedora-commons-users@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users >>>>> >>>> ------------------------------------------------------------------------------ >>>> Achieve unprecedented app performance and reliability >>>> What every C/C++ and Fortran developer should know. >>>> Learn how Intel has extended the reach of its next-generation tools >>>> to help boost performance applications - inlcuding clusters. >>>> http://p.sf.net/sfu/intel-dev2devmay >>>> _______________________________________________ >>>> Fedora-commons-users mailing list >>>> Fedora-commons-users@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users >> >> ------------------------------------------------------------------------------ >> Achieve unprecedented app performance and reliability >> What every C/C++ and Fortran developer should know. >> Learn how Intel has extended the reach of its next-generation tools >> to help boost performance applications - inlcuding clusters. >> http://p.sf.net/sfu/intel-dev2devmay >> _______________________________________________ >> Fedora-commons-users mailing list >> Fedora-commons-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > ------------------------------------------------------------------------------ Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
