I think the ZipFileDataFormat was recently improved to use stream caching so it spool to disk for big streams.
Ticket is: CAMEL-9274 On Thu, Nov 24, 2016 at 9:44 AM, Siano, Stephan <[email protected]> wrote: > Hi, > > The ZipFileDataFormat reads the file content into a byte array, I guess your > server goes OOM while doing that. Theoretically it might be possible to > extend the component to support streaming if a stream caching is enabled on > the route, however someone would have to contribute that to Camel. > > Short term the only way I see for you is to increase the heap size of your > server. 45 MB is not so much memory, so this should be feasible. > > Best regards > > Stephan > > -----Original Message----- > From: adalfovo [mailto:[email protected]] > Sent: Mittwoch, 23. November 2016 10:41 > To: [email protected] > Subject: Unzip huge single entry file throws OutOfMemory > > Hi, > we have a 3.5 Mb zip file that contains a 45Mb text file. When we try to > unzip it the route throws an OutOfMemory. > Camel runs on a Jboss EAP 6.4.5 instance. > This is our route : > > from("servlet://?httpMethodRestrict=POST&servletName=XXX&matchOnUriPrefix=true") > .unmarshal().mimeMultipart() > .unmarshal().zipFile().... > > The stacktrace says: > > --------------------------------------------------------------------------------------------------------------------------------------- > RouteId ProcessorId Processor > Elapsed (ms) > [route1] [post] > [servlet:/?httpMethodRestrict=POST&matchOnUriPrefix=true&servletName=XXX] [ > 1008] > [route1] [unmarshal1] > [unmarshal[org.apache.camel.model.dataformat.MimeMultipartDataFormat@5bbe2866] > ] [ 0] > [route1] [unmarshal2] > [unmarshal[org.apache.camel.model.dataformat.ZipFileDataFormat@1ebcbd75] > ] [ 1006] > > Stacktrace > --------------------------------------------------------------------------------------------------------------------------------------- > org.apache.camel.CamelExecutionException: Exception occurred during execution > on the exchange: Exchange[ID-123] > at > org.apache.camel.util.ObjectHelper.wrapCamelExecutionException(ObjectHelper.java:1706) > at > org.apache.camel.impl.DefaultExchange.setException(DefaultExchange.java:349) > at > org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:84) > at > org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) > at > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:468) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) > at org.apache.camel.processor.Pipeline.process(Pipeline.java:121) > at org.apache.camel.processor.Pipeline.process(Pipeline.java:83) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) > at > org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109) > at > org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:91) > at > org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:143) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at > org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:854) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.OutOfMemoryError: Java heap space > > > How can we fix this problem? > > > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Unzip-huge-single-entry-file-throws-OutOfMemory-tp5790404.html > Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
