Hi, The ZipFileDataFormat reads the file content into a byte array, I guess your server goes OOM while doing that. Theoretically it might be possible to extend the component to support streaming if a stream caching is enabled on the route, however someone would have to contribute that to Camel.
Short term the only way I see for you is to increase the heap size of your server. 45 MB is not so much memory, so this should be feasible. Best regards Stephan -----Original Message----- From: adalfovo [mailto:[email protected]] Sent: Mittwoch, 23. November 2016 10:41 To: [email protected] Subject: Unzip huge single entry file throws OutOfMemory Hi, we have a 3.5 Mb zip file that contains a 45Mb text file. When we try to unzip it the route throws an OutOfMemory. Camel runs on a Jboss EAP 6.4.5 instance. This is our route : from("servlet://?httpMethodRestrict=POST&servletName=XXX&matchOnUriPrefix=true") .unmarshal().mimeMultipart() .unmarshal().zipFile().... The stacktrace says: --------------------------------------------------------------------------------------------------------------------------------------- RouteId ProcessorId Processor Elapsed (ms) [route1] [post] [servlet:/?httpMethodRestrict=POST&matchOnUriPrefix=true&servletName=XXX] [ 1008] [route1] [unmarshal1] [unmarshal[org.apache.camel.model.dataformat.MimeMultipartDataFormat@5bbe2866] ] [ 0] [route1] [unmarshal2] [unmarshal[org.apache.camel.model.dataformat.ZipFileDataFormat@1ebcbd75] ] [ 1006] Stacktrace --------------------------------------------------------------------------------------------------------------------------------------- org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-123] at org.apache.camel.util.ObjectHelper.wrapCamelExecutionException(ObjectHelper.java:1706) at org.apache.camel.impl.DefaultExchange.setException(DefaultExchange.java:349) at org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:84) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:468) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) at org.apache.camel.processor.Pipeline.process(Pipeline.java:121) at org.apache.camel.processor.Pipeline.process(Pipeline.java:83) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:91) at org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:143) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:854) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.OutOfMemoryError: Java heap space How can we fix this problem? -- View this message in context: http://camel.465427.n5.nabble.com/Unzip-huge-single-entry-file-throws-OutOfMemory-tp5790404.html Sent from the Camel - Users mailing list archive at Nabble.com.
