I can add a description that parameter is read twice. 2016-10-20 13:34 GMT+03:00 Vova Shelgunov <[email protected]>:
> *Tomohisa*, actually initially I posted a bug in Jira ( > https://issues.apache.org/jira/browse/CAMEL-10399), but Claus closed it, > can I reopen it? > > 2016-10-20 10:49 GMT+03:00 Tomohisa Igarashi <[email protected]>: > >> Hmm that smells like a bug in DefaultHttpBinding.populateRequestParameters(), >> it shouldn't read a parameter value which is already handled in >> populateAttachments(). Would you file a JIRA for it? >> >> On 10/20/2016 03:18 PM, Vova Shelgunov wrote: >> >>> Hi, >>> >>> I tried to debug populateRequestParameters in DefaultHttpBinding and >>> noticed that request parameters contain file (with byte[] array value) that >>> I am sending to endpoint: >>> >>> >>> >>> >>> It is a reason why I have OOM error. >>> >>> Let me note that I am sending file using google http client: >>> >>> >>> I attached MultipartFormDataContent. >>> >>> As for allowJavaSerializedObject I did not change default value (it is >>> false). >>> >>> 2016-10-20 4:24 GMT+03:00 Tomohisa Igarashi <[email protected] >>> <mailto:[email protected]>>: >>> >>> Hi, >>> >>> It seems like the OOM occurs when it's mapping request parameters, >>> but not request body which is passed as an InputStream by default. Do you >>> have any large parameter other than body? >>> Note that if you enable allowJavaSerializedObject, then body is >>> deseriallized as a Java Object, may cause OOM. >>> >>> Thanks, >>> Tomo >>> >>> >>> On 10/19/2016 09:39 PM, Vova Shelgunov wrote: >>> >>> Hi Andrea, >>> >>> Yes, I run application using jar file. Java has 1G heap. >>> >>> Actually I assumed that camel will not read file in single >>> string, but will >>> use InputStream to read the data. >>> >>> 2016-10-19 15:28 GMT+03:00 Andrea Cosentino >>> <[email protected]>: >>> >>> Is the camel-context deployed as a simple JAR? What are you >>> using inside >>> the Debian container? How much memory has your container? >>> Did you set JVM_OPTS witch specific values for max Heap size >>> and min Heap >>> size? >>> >>> -- >>> Andrea Cosentino >>> ---------------------------------- >>> Apache Camel PMC Member >>> Apache Karaf Committer >>> Apache Servicemix Committer >>> Email: [email protected] <mailto:[email protected]> >>> Twitter: @oscerd2 >>> Github: oscerd >>> >>> >>> >>> On Wednesday, October 19, 2016 2:00 PM, Vova Shelgunov < >>> [email protected] <mailto:[email protected]>> >>> >>> wrote: >>> Hi, >>> >>> I catched this OutOfMemoryError when I sent a 250 MB file to >>> jetty >>> endpoint: >>> >>> java.lang.OutOfMemoryError: Java heap space >>> at java.lang.StringCoding.decode( >>> StringCoding.java:215) >>> at java.lang.String.<init>(String.java:463) >>> at java.lang.String.<init>(String.java:515) >>> at >>> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper. >>> getParameterBytesAsString(MultiPartFilter.java:390) >>> at >>> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.getParame >>> terValues( >>> MultiPartFilter.java:337) >>> at >>> org.apache.camel.http.common.DefaultHttpBinding.populateRequ >>> estParameters( >>> DefaultHttpBinding.java:219) >>> at >>> org.apache.camel.http.common.DefaultHttpBinding.readHeaders( >>> DefaultHttpBinding.java:174) >>> at >>> org.apache.camel.http.common.DefaultHttpBinding.readRequest( >>> DefaultHttpBinding.java:110) >>> at >>> org.apache.camel.http.common.HttpMessage.<init>(HttpMessage. >>> java:52) >>> at >>> org.apache.camel.component.jetty.CamelContinuationServlet.do >>> <http://ty.CamelContinuationServlet.do>Service( >>> >>> CamelContinuationServlet.java:161) >>> at >>> org.apache.camel.http.common.CamelServlet.service(CamelServl >>> et.java:74) >>> at javax.servlet.http.HttpServlet >>> .service(HttpServlet.java:790) >>> at >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder >>> .java:812) >>> at >>> org.eclipse.jetty.servlet.ServletHandler$CachedChain. >>> doFilter(ServletHandler.java:1669) >>> at >>> org.eclipse.jetty.servlets.MultiPartFilter.doFilter( >>> MultiPartFilter.java:200) >>> at >>> org.apache.camel.component.jetty.CamelFilterWrapper. >>> doFilter(CamelFilterWrapper.java:43) >>> at >>> org.eclipse.jetty.servlet.ServletHandler$CachedChain. >>> doFilter(ServletHandler.java:1652) >>> at >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan >>> dler.java:585) >>> at >>> org.eclipse.jetty.server.handler.ContextHandler. >>> doHandle(ContextHandler.java:1127) >>> at >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand >>> ler.java:515) >>> at >>> org.eclipse.jetty.server.handler.ContextHandler. >>> doScope(ContextHandler.java:1061) >>> at >>> org.eclipse.jetty.server.handler.ScopedHandler.handle( >>> ScopedHandler.java:141) >>> at >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle( >>> HandlerWrapper.java:97) >>> at org.eclipse.jetty.server.Serve >>> r.handle(Server.java:499) >>> at org.eclipse.jetty.server.HttpC >>> hannel.handle(HttpChannel.java:311) >>> at >>> org.eclipse.jetty.server.HttpConnection.onFillable( >>> HttpConnection.java:257) >>> at >>> org.eclipse.jetty.io <http://org.eclipse.jetty.io>. >>> AbstractConnection$2.run(AbstractConnection.java:544) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( >>> QueuedThreadPool.java:635) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( >>> QueuedThreadPool.java:555) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> My configuration: >>> >>> <route id="ds-rest-archive-upload-processor"> >>> <from uri="jetty: >>> https://storage:8443/data/archive?sslContextParametersRef= < >>> https://storage:8443/data/archive?sslContextParametersRef=> >>> sslContextParameters&requestBufferSize=32768&responseBufferS >>> ize=32768& >>> httpMethodRestrict=POST >>> <https://storage:8443/data/archive?sslContextParametersRef= >>> <https://storage:8443/data/archive?sslContextParametersRef=> >>> >>> sslContextParameters&requestBufferSize=32768&responseBufferS >>> ize=32768& >>> httpMethodRestrict=POST>" >>> /> >>> >>> <to uri="bean:uploadFromArchiveFileProcessor?method=process" >>> pattern="InOut" /> >>> >>> </route> >>> >>> <bean id="jetty" >>> class="org.apache.camel.component.jetty9.JettyHttpComponent9 >>> "> >>> <property name="endpointClass" >>> value="org.apache.camel.component.jetty9.JettyHttpEndpoint9" >>> /> >>> <property name="sslContextParameters" >>> ref="sslContextParameters" /> >>> <property name="useContinuation" value="true" /> >>> <property name="requestBufferSize" value="32768" /> >>> <property name="responseBufferSize" value="32768" /> >>> >>> <property name="threadPool"> >>> <bean class="org.eclipse.jetty.util. >>> thread.QueuedThreadPool"> >>> <constructor-arg index="0" value="1000" /> >>> </bean> >>> </property> >>> >>> <property name="jettyHttpBinding"> >>> <bean class="org.apache.camel.compon >>> ent.jetty.DefaultJettyHttpBinding"> >>> <property name="transferException" value="false" /> >>> </bean> >>> </property> >>> </bean> >>> >>> My camel application is running in docker container with >>> limited amount of >>> memory, and I did not expect, that a file will be read a >>> string instead of >>> reading from InputStream. >>> >>> Is there any way to consume large files with small amount of >>> memory? >>> >>> >>> Regards, >>> >>> Uladzimir >>> >>> >>> >>> >
