Hi Janna I've added a note to the page http://www.fedora-commons.org/confluence/display/FCR30/Client+Command-line+U tilities as you've suggested. In fact it would be a fairly minor change to (a) use FEDORA_OPTS if it is set, otherwise (b) use JAVA_OPTS if it is set otherwise (c) use defaults (it is what the scripts currently do for other environment variables). However if it's not an issue for you I'll not raise a JIRA issue for this - unless anyone else feels this is worthwhile. It does look like memory handling for large objects may be an issue in any case for this script, but again I'll leave it for others to bring up as an issue if it affects them. Thanks for testing this Regards Steve
-----Original Message----- From: Janna Wemekamp [mailto:[email protected]] Sent: 18 May 2010 23:50 To: Steve Bayliss Cc: 'Fedora Users' Subject: Re: [Fedora-commons-users] OutOfMemoryError in client fedora-export but not in REST API's export (archive context) Thanks Steve! To create an archive export of the C4oc:4629 (which has a 3M PDF datastream), I had to bump the -X options in client/bin/env-client.sh to -Xms64m -Xmx256m. The archive file was 4.7M. However, when I try to export a much larger object which has a 163M PDF datastream, I've had no success even with -Xms128m -Xmx8g. (And since my server has only 8GB of physical memory, I'm not going to raise this!) Using the REST API, I set the -X options in $JAVA_OPTS to -Xms128m -Xmx3g and successfully exported the larger object. It has an archive size of 259M. It might be worth increasing the defaults in env-client.sh somewhat. It would also be useful to add a note to the doco that the client utilities don't use the server's JAVA_OPTS parameters and explain how to increase the Java heap size. Personally, I see no need for a more sophisticated mechanism for controlling memory in the client scripts than simply editing env-client.sh. As for the failure on the large object using the client utility - I'm not fussed. I'm happy to use the REST API instead. Cheers Janna On 18/05/2010 16:25, Steve Bayliss wrote: Hi Janna The client script memory settings are defined in env-client.sh (in the same directory as fedora-export.sh) - towards the end of this file you'll see a line exec_cmd="exec \"$java\" -Xms64m -Xmx96m \ Setting $JAVA_OPTS won't have any effect. You could try increasing -Xms and -Xmx in this script. Let us know what values work for you - it sounds like we should either increase these defaults, or provide some mechanism for controlling memory for client scripts? Regards Steve -----Original Message----- From: Janna Wemekamp [mailto:[email protected]] Sent: 18 May 2010 00:09 To: Fedora Users Subject: [Fedora-commons-users] OutOfMemoryError in client fedora-export but not in REST API's export (archive context) G'day, When I run the client script fedora-export.sh and specify the PID of an object containing three managed streams one of which is a 3MB PDF file, the export fails with an OutOfMemoryError: Java heap space exception. The error is only reported on the command-line and is NOT present in the Fedora server log. bin/fedora-export.sh dev.fedapps.toldark.com.au:80 fedoraAdmin xxx c4oc:4629 'info:fedora/fedora-system:FOXML-1.1' archive /tmp/fc/export/dev-full http Exporting c4oc:4629 to /tmp/fc/export/dev-full/c4oc_4629.xml Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:10 0) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) at java.lang.StringBuffer.append(StringBuffer.java:224) at org.apache.xerces.dom.DeferredDocumentImpl.getNodeValueString(Unknown Source) at org.apache.xerces.dom.DeferredDocumentImpl.getNodeValueString(Unknown Source) at org.apache.xerces.dom.DeferredTextImpl.synchronizeData(Unknown Source) at org.apache.xerces.dom.CharacterDataImpl.getNodeValue(Unknown Source) at com.sun.org.apache.xml.internal.serialize.BaseMarkupSerializer.serializeNode (BaseMarkupSerializer.java:1023) at com.sun.org.apache.xml.internal.serialize.XMLSerializer.serializeElement(XML Serializer.java:1068) at com.sun.org.apache.xml.internal.serialize.BaseMarkupSerializer.serializeNode (BaseMarkupSerializer.java:1190) at com.sun.org.apache.xml.internal.serialize.XMLSerializer.serializeElement(XML Serializer.java:1068) at com.sun.org.apache.xml.internal.serialize.BaseMarkupSerializer.serializeNode (BaseMarkupSerializer.java:1190) at com.sun.org.apache.xml.internal.serialize.XMLSerializer.serializeElement(XML Serializer.java:1068) at com.sun.org.apache.xml.internal.serialize.BaseMarkupSerializer.serializeNode (BaseMarkupSerializer.java:1190) at com.sun.org.apache.xml.internal.serialize.XMLSerializer.serializeElement(XML Serializer.java:1068) at com.sun.org.apache.xml.internal.serialize.BaseMarkupSerializer.serializeNode (BaseMarkupSerializer.java:1190) at com.sun.org.apache.xml.internal.serialize.BaseMarkupSerializer.serializeNode (BaseMarkupSerializer.java:1265) at com.sun.org.apache.xml.internal.serialize.BaseMarkupSerializer.serialize(Bas eMarkupSerializer.java:468) at fedora.client.utility.export.AutoExporter.export(AutoExporter.java:148) at fedora.client.utility.export.Export.one(Export.java:84) at fedora.client.utility.export.Export.main(Export.java:281) Bumping up the Java heap size in $JAVA_OPTS or using a different Java version has no effect; the same error occurs. Environment: RHEL 5.5, Java 1.6.0_20, Fedora Commons 3.3 However, if I use the REST API to export the same object in archive context, there's no problem. I can also export much larger objects successfully although I do have to bump up the Java heap space. Perhaps there's a bug in the client export code? Janna -- Janna Wemekamp Toldark Pty Limited
------------------------------------------------------------------------------
_______________________________________________ Fedora-commons-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
