Re: [xwiki-users] Can't make big uploads?
When saving an attachment XWikiAttachment.updateContentArchive produce a base 64 String of the attachment (so a lot bigger than the attachment size) before copy it again as String[] (so again double the size in memory) and then sending it to the database (which is not quite the streamed thing you would expect in case of MySQL for example). There is thing that can be improved (but not much we can do about non streaming MySQL connector) but yes currently default attachment store is not exactly great to store 600 Mb files unless you allocate a looot of memory to your XWiki instance. Filesystem attachment should support it well (been made just for this kind of use case after all), see http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystemAttachmentStore. On Sat, Jan 10, 2015 at 12:53 AM, Paul Libbrecht p...@hoplahup.net wrote: Hello all, after I changed my max upload size to 2Gb, and after having used the filesystem attachment, I thought it would be flawless to upload a 600 Mb but no. I get the following exception: Caused by: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2786) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94) at org.apache.commons.io.output.ProxyOutputStream.write(ProxyOutputStream.java:90) at org.apache.commons.codec.binary.BaseNCodecOutputStream.flush(BaseNCodecOutputStream.java:116) at org.apache.commons.codec.binary.BaseNCodecOutputStream.write(BaseNCodecOutputStream.java:97) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1720) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1696) at org.apache.commons.io.IOUtils.copy(IOUtils.java:1671) at com.xpn.xwiki.internal.xml.XMLWriter.writeBase64(XMLWriter.java:182) at com.xpn.xwiki.doc.XWikiAttachment.toXML(XWikiAttachment.java:413) at com.xpn.xwiki.doc.XWikiAttachment.toStringXML(XWikiAttachment.java:340) at com.xpn.xwiki.doc.XWikiAttachmentArchive.updateArchive(XWikiAttachmentArchive.java:180) at com.xpn.xwiki.doc.XWikiAttachment.updateContentArchive(XWikiAttachment.java:718) at com.xpn.xwiki.store.XWikiHibernateStore.saveAttachment(XWikiHibernateStore.java:1528) at com.xpn.xwiki.store.XWikiHibernateStore.saveAttachmentList(XWikiHibernateStore.java:1496) Could it be that this copy is indeed copying the whole file? paul ___ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users -- Thomas Mortagne ___ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
Re: [xwiki-users] Can't make big uploads?
Thomas, I believe I am using the filesystem attachment. At least that's when I thought when using the following configuration: # enable attachment filesystem storage xwiki.store.attachment.hint=file xwiki.store.attachment.versioning.hint=file xwiki.store.attachment.recyclebin.hint=file Any other way for me to proof? thanks Paul On 12 janv. 2015, at 09:38, Thomas Mortagne thomas.morta...@xwiki.com wrote: When saving an attachment XWikiAttachment.updateContentArchive produce a base 64 String of the attachment (so a lot bigger than the attachment size) before copy it again as String[] (so again double the size in memory) and then sending it to the database (which is not quite the streamed thing you would expect in case of MySQL for example). There is thing that can be improved (but not much we can do about non streaming MySQL connector) but yes currently default attachment store is not exactly great to store 600 Mb files unless you allocate a looot of memory to your XWiki instance. Filesystem attachment should support it well (been made just for this kind of use case after all), see http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystemAttachmentStore. ___ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
Re: [xwiki-users] Can't make big uploads?
Well that stack trace don't look like filesystem attachment storage stack trace. Attachment archives are handled by ListAttachmentArchive so we should have seen a ListAttachmentArchive#updateArchive call instead of the XWikiAttachment.updateContentArchive call. I agree that your configuration looks right so not sure why you don't end up in the filesystem attachment store code. On Mon, Jan 12, 2015 at 10:05 AM, Paul Libbrecht p...@hoplahup.net wrote: Thomas, I believe I am using the filesystem attachment. At least that's when I thought when using the following configuration: # enable attachment filesystem storage xwiki.store.attachment.hint=file xwiki.store.attachment.versioning.hint=file xwiki.store.attachment.recyclebin.hint=file Any other way for me to proof? thanks Paul On 12 janv. 2015, at 09:38, Thomas Mortagne thomas.morta...@xwiki.com wrote: When saving an attachment XWikiAttachment.updateContentArchive produce a base 64 String of the attachment (so a lot bigger than the attachment size) before copy it again as String[] (so again double the size in memory) and then sending it to the database (which is not quite the streamed thing you would expect in case of MySQL for example). There is thing that can be improved (but not much we can do about non streaming MySQL connector) but yes currently default attachment store is not exactly great to store 600 Mb files unless you allocate a looot of memory to your XWiki instance. Filesystem attachment should support it well (been made just for this kind of use case after all), see http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystemAttachmentStore. ___ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users -- Thomas Mortagne ___ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
Re: [xwiki-users] Can't make big uploads?
The error makes it look like FSAttach is disabled. Check for files existing in the Permanent Storage directory in the server's filesystem. Thanks, Caleb On 01/12/2015 10:05 AM, Paul Libbrecht wrote: Thomas, I believe I am using the filesystem attachment. At least that's when I thought when using the following configuration: # enable attachment filesystem storage xwiki.store.attachment.hint=file xwiki.store.attachment.versioning.hint=file xwiki.store.attachment.recyclebin.hint=file Any other way for me to proof? thanks Paul On 12 janv. 2015, at 09:38, Thomas Mortagne thomas.morta...@xwiki.com wrote: When saving an attachment XWikiAttachment.updateContentArchive produce a base 64 String of the attachment (so a lot bigger than the attachment size) before copy it again as String[] (so again double the size in memory) and then sending it to the database (which is not quite the streamed thing you would expect in case of MySQL for example). There is thing that can be improved (but not much we can do about non streaming MySQL connector) but yes currently default attachment store is not exactly great to store 600 Mb files unless you allocate a looot of memory to your XWiki instance. Filesystem attachment should support it well (been made just for this kind of use case after all), see http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Attachments#HFilesystemAttachmentStore. ___ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users ___ users mailing list users@xwiki.org http://lists.xwiki.org/mailman/listinfo/users
