Hmm that smells like a bug in DefaultHttpBinding.populateRequestParameters(), it shouldn't read a parameter value which is already handled in populateAttachments(). Would you file a JIRA for it?
On 10/20/2016 03:18 PM, Vova Shelgunov wrote:
Hi, I tried to debug populateRequestParameters in DefaultHttpBinding and noticed that request parameters contain file (with byte[] array value) that I am sending to endpoint: It is a reason why I have OOM error. Let me note that I am sending file using google http client: I attached MultipartFormDataContent. As for allowJavaSerializedObject I did not change default value (it is false). 2016-10-20 4:24 GMT+03:00 Tomohisa Igarashi <[email protected] <mailto:[email protected]>>: Hi, It seems like the OOM occurs when it's mapping request parameters, but not request body which is passed as an InputStream by default. Do you have any large parameter other than body? Note that if you enable allowJavaSerializedObject, then body is deseriallized as a Java Object, may cause OOM. Thanks, Tomo On 10/19/2016 09:39 PM, Vova Shelgunov wrote: Hi Andrea, Yes, I run application using jar file. Java has 1G heap. Actually I assumed that camel will not read file in single string, but will use InputStream to read the data. 2016-10-19 15:28 GMT+03:00 Andrea Cosentino <[email protected]>: Is the camel-context deployed as a simple JAR? What are you using inside the Debian container? How much memory has your container? Did you set JVM_OPTS witch specific values for max Heap size and min Heap size? -- Andrea Cosentino ---------------------------------- Apache Camel PMC Member Apache Karaf Committer Apache Servicemix Committer Email: [email protected] <mailto:[email protected]> Twitter: @oscerd2 Github: oscerd On Wednesday, October 19, 2016 2:00 PM, Vova Shelgunov <[email protected] <mailto:[email protected]>> wrote: Hi, I catched this OutOfMemoryError when I sent a 250 MB file to jetty endpoint: java.lang.OutOfMemoryError: Java heap space at java.lang.StringCoding.decode(StringCoding.java:215) at java.lang.String.<init>(String.java:463) at java.lang.String.<init>(String.java:515) at org.eclipse.jetty.servlets.MultiPartFilter$Wrapper. getParameterBytesAsString(MultiPartFilter.java:390) at org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.getParameterValues( MultiPartFilter.java:337) at org.apache.camel.http.common.DefaultHttpBinding.populateRequestParameters( DefaultHttpBinding.java:219) at org.apache.camel.http.common.DefaultHttpBinding.readHeaders( DefaultHttpBinding.java:174) at org.apache.camel.http.common.DefaultHttpBinding.readRequest( DefaultHttpBinding.java:110) at org.apache.camel.http.common.HttpMessage.<init>(HttpMessage.java:52) at org.apache.camel.component.jetty.CamelContinuationServlet.do <http://ty.CamelContinuationServlet.do>Service( CamelContinuationServlet.java:161) at org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain. doFilter(ServletHandler.java:1669) at org.eclipse.jetty.servlets.MultiPartFilter.doFilter( MultiPartFilter.java:200) at org.apache.camel.component.jetty.CamelFilterWrapper. doFilter(CamelFilterWrapper.java:43) at org.eclipse.jetty.servlet.ServletHandler$CachedChain. doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ContextHandler. doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.handler.ContextHandler. doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle( ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle( HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable( HttpConnection.java:257) at org.eclipse.jetty.io <http://org.eclipse.jetty.io>.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) My configuration: <route id="ds-rest-archive-upload-processor"> <from uri="jetty: https://storage:8443/data/archive?sslContextParametersRef= <https://storage:8443/data/archive?sslContextParametersRef=> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768& httpMethodRestrict=POST <https://storage:8443/data/archive?sslContextParametersRef= <https://storage:8443/data/archive?sslContextParametersRef=> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768& httpMethodRestrict=POST>" /> <to uri="bean:uploadFromArchiveFileProcessor?method=process" pattern="InOut" /> </route> <bean id="jetty" class="org.apache.camel.component.jetty9.JettyHttpComponent9"> <property name="endpointClass" value="org.apache.camel.component.jetty9.JettyHttpEndpoint9" /> <property name="sslContextParameters" ref="sslContextParameters" /> <property name="useContinuation" value="true" /> <property name="requestBufferSize" value="32768" /> <property name="responseBufferSize" value="32768" /> <property name="threadPool"> <bean class="org.eclipse.jetty.util.thread.QueuedThreadPool"> <constructor-arg index="0" value="1000" /> </bean> </property> <property name="jettyHttpBinding"> <bean class="org.apache.camel.component.jetty.DefaultJettyHttpBinding"> <property name="transferException" value="false" /> </bean> </property> </bean> My camel application is running in docker container with limited amount of memory, and I did not expect, that a file will be read a string instead of reading from InputStream. Is there any way to consume large files with small amount of memory? Regards, Uladzimir
