Hi, I tried to debug populateRequestParameters in DefaultHttpBinding and noticed that request parameters contain file (with byte[] array value) that I am sending to endpoint:
It is a reason why I have OOM error. Let me note that I am sending file using google http client: I attached MultipartFormDataContent. As for allowJavaSerializedObject I did not change default value (it is false). 2016-10-20 4:24 GMT+03:00 Tomohisa Igarashi <[email protected]>: > Hi, > > It seems like the OOM occurs when it's mapping request parameters, but not > request body which is passed as an InputStream by default. Do you have any > large parameter other than body? > Note that if you enable allowJavaSerializedObject, then body is > deseriallized as a Java Object, may cause OOM. > > Thanks, > Tomo > > > On 10/19/2016 09:39 PM, Vova Shelgunov wrote: > >> Hi Andrea, >> >> Yes, I run application using jar file. Java has 1G heap. >> >> Actually I assumed that camel will not read file in single string, but >> will >> use InputStream to read the data. >> >> 2016-10-19 15:28 GMT+03:00 Andrea Cosentino <[email protected] >> >: >> >> Is the camel-context deployed as a simple JAR? What are you using inside >>> the Debian container? How much memory has your container? >>> Did you set JVM_OPTS witch specific values for max Heap size and min Heap >>> size? >>> >>> -- >>> Andrea Cosentino >>> ---------------------------------- >>> Apache Camel PMC Member >>> Apache Karaf Committer >>> Apache Servicemix Committer >>> Email: [email protected] >>> Twitter: @oscerd2 >>> Github: oscerd >>> >>> >>> >>> On Wednesday, October 19, 2016 2:00 PM, Vova Shelgunov <[email protected] >>> > >>> wrote: >>> Hi, >>> >>> I catched this OutOfMemoryError when I sent a 250 MB file to jetty >>> endpoint: >>> >>> java.lang.OutOfMemoryError: Java heap space >>> at java.lang.StringCoding.decode(StringCoding.java:215) >>> at java.lang.String.<init>(String.java:463) >>> at java.lang.String.<init>(String.java:515) >>> at >>> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper. >>> getParameterBytesAsString(MultiPartFilter.java:390) >>> at >>> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.getParameterValues( >>> MultiPartFilter.java:337) >>> at >>> org.apache.camel.http.common.DefaultHttpBinding.populateRequ >>> estParameters( >>> DefaultHttpBinding.java:219) >>> at >>> org.apache.camel.http.common.DefaultHttpBinding.readHeaders( >>> DefaultHttpBinding.java:174) >>> at >>> org.apache.camel.http.common.DefaultHttpBinding.readRequest( >>> DefaultHttpBinding.java:110) >>> at >>> org.apache.camel.http.common.HttpMessage.<init>(HttpMessage.java:52) >>> at >>> org.apache.camel.component.jetty.CamelContinuationServlet.doService( >>> CamelContinuationServlet.java:161) >>> at >>> org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) >>> at >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) >>> at >>> org.eclipse.jetty.servlet.ServletHandler$CachedChain. >>> doFilter(ServletHandler.java:1669) >>> at >>> org.eclipse.jetty.servlets.MultiPartFilter.doFilter( >>> MultiPartFilter.java:200) >>> at >>> org.apache.camel.component.jetty.CamelFilterWrapper. >>> doFilter(CamelFilterWrapper.java:43) >>> at >>> org.eclipse.jetty.servlet.ServletHandler$CachedChain. >>> doFilter(ServletHandler.java:1652) >>> at >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan >>> dler.java:585) >>> at >>> org.eclipse.jetty.server.handler.ContextHandler. >>> doHandle(ContextHandler.java:1127) >>> at >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand >>> ler.java:515) >>> at >>> org.eclipse.jetty.server.handler.ContextHandler. >>> doScope(ContextHandler.java:1061) >>> at >>> org.eclipse.jetty.server.handler.ScopedHandler.handle( >>> ScopedHandler.java:141) >>> at >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle( >>> HandlerWrapper.java:97) >>> at org.eclipse.jetty.server.Server.handle(Server.java:499) >>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel. >>> java:311) >>> at >>> org.eclipse.jetty.server.HttpConnection.onFillable( >>> HttpConnection.java:257) >>> at >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnec >>> tion.java:544) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( >>> QueuedThreadPool.java:635) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( >>> QueuedThreadPool.java:555) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> My configuration: >>> >>> <route id="ds-rest-archive-upload-processor"> >>> <from uri="jetty: >>> https://storage:8443/data/archive?sslContextParametersRef= >>> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768& >>> httpMethodRestrict=POST >>> <https://storage:8443/data/archive?sslContextParametersRef= >>> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768& >>> httpMethodRestrict=POST>" >>> /> >>> >>> <to uri="bean:uploadFromArchiveFileProcessor?method=process" >>> pattern="InOut" /> >>> >>> </route> >>> >>> <bean id="jetty" >>> class="org.apache.camel.component.jetty9.JettyHttpComponent9"> >>> <property name="endpointClass" >>> value="org.apache.camel.component.jetty9.JettyHttpEndpoint9" /> >>> <property name="sslContextParameters" ref="sslContextParameters" /> >>> <property name="useContinuation" value="true" /> >>> <property name="requestBufferSize" value="32768" /> >>> <property name="responseBufferSize" value="32768" /> >>> >>> <property name="threadPool"> >>> <bean class="org.eclipse.jetty.util.thread.QueuedThreadPool"> >>> <constructor-arg index="0" value="1000" /> >>> </bean> >>> </property> >>> >>> <property name="jettyHttpBinding"> >>> <bean class="org.apache.camel.component.jetty.DefaultJettyHttpBinding"> >>> <property name="transferException" value="false" /> >>> </bean> >>> </property> >>> </bean> >>> >>> My camel application is running in docker container with limited amount >>> of >>> memory, and I did not expect, that a file will be read a string instead >>> of >>> reading from InputStream. >>> >>> Is there any way to consume large files with small amount of memory? >>> >>> >>> Regards, >>> >>> Uladzimir >>> >>> >>
