Yes if you can see file content is actually loaded in
populateRequestParameters(), I think it's likely to be a bug. Please make sure
it's not just a file name, and update the description with what you found. If
you can attach a minimal reproducer project that would be better.
On 10/20/2016 07:35 PM, Vova Shelgunov wrote:
I can add a description that parameter is read twice.
2016-10-20 13:34 GMT+03:00 Vova Shelgunov <[email protected]>:
*Tomohisa*, actually initially I posted a bug in Jira (
https://issues.apache.org/jira/browse/CAMEL-10399), but Claus closed it,
can I reopen it?
2016-10-20 10:49 GMT+03:00 Tomohisa Igarashi <[email protected]>:
Hmm that smells like a bug in DefaultHttpBinding.populateRequestParameters(),
it shouldn't read a parameter value which is already handled in
populateAttachments(). Would you file a JIRA for it?
On 10/20/2016 03:18 PM, Vova Shelgunov wrote:
Hi,
I tried to debug populateRequestParameters in DefaultHttpBinding and
noticed that request parameters contain file (with byte[] array value) that
I am sending to endpoint:
It is a reason why I have OOM error.
Let me note that I am sending file using google http client:
I attached MultipartFormDataContent.
As for allowJavaSerializedObject I did not change default value (it is
false).
2016-10-20 4:24 GMT+03:00 Tomohisa Igarashi <[email protected]
<mailto:[email protected]>>:
Hi,
It seems like the OOM occurs when it's mapping request parameters,
but not request body which is passed as an InputStream by default. Do you
have any large parameter other than body?
Note that if you enable allowJavaSerializedObject, then body is
deseriallized as a Java Object, may cause OOM.
Thanks,
Tomo
On 10/19/2016 09:39 PM, Vova Shelgunov wrote:
Hi Andrea,
Yes, I run application using jar file. Java has 1G heap.
Actually I assumed that camel will not read file in single
string, but will
use InputStream to read the data.
2016-10-19 15:28 GMT+03:00 Andrea Cosentino
<[email protected]>:
Is the camel-context deployed as a simple JAR? What are you
using inside
the Debian container? How much memory has your container?
Did you set JVM_OPTS witch specific values for max Heap size
and min Heap
size?
--
Andrea Cosentino
----------------------------------
Apache Camel PMC Member
Apache Karaf Committer
Apache Servicemix Committer
Email: [email protected] <mailto:[email protected]>
Twitter: @oscerd2
Github: oscerd
On Wednesday, October 19, 2016 2:00 PM, Vova Shelgunov <
[email protected] <mailto:[email protected]>>
wrote:
Hi,
I catched this OutOfMemoryError when I sent a 250 MB file to
jetty
endpoint:
java.lang.OutOfMemoryError: Java heap space
at java.lang.StringCoding.decode(
StringCoding.java:215)
at java.lang.String.<init>(String.java:463)
at java.lang.String.<init>(String.java:515)
at
org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.
getParameterBytesAsString(MultiPartFilter.java:390)
at
org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.getParame
terValues(
MultiPartFilter.java:337)
at
org.apache.camel.http.common.DefaultHttpBinding.populateRequ
estParameters(
DefaultHttpBinding.java:219)
at
org.apache.camel.http.common.DefaultHttpBinding.readHeaders(
DefaultHttpBinding.java:174)
at
org.apache.camel.http.common.DefaultHttpBinding.readRequest(
DefaultHttpBinding.java:110)
at
org.apache.camel.http.common.HttpMessage.<init>(HttpMessage.
java:52)
at
org.apache.camel.component.jetty.CamelContinuationServlet.do
<http://ty.CamelContinuationServlet.do>Service(
CamelContinuationServlet.java:161)
at
org.apache.camel.http.common.CamelServlet.service(CamelServl
et.java:74)
at javax.servlet.http.HttpServlet
.service(HttpServlet.java:790)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder
.java:812)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.
doFilter(ServletHandler.java:1669)
at
org.eclipse.jetty.servlets.MultiPartFilter.doFilter(
MultiPartFilter.java:200)
at
org.apache.camel.component.jetty.CamelFilterWrapper.
doFilter(CamelFilterWrapper.java:43)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.
doFilter(ServletHandler.java:1652)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan
dler.java:585)
at
org.eclipse.jetty.server.handler.ContextHandler.
doHandle(ContextHandler.java:1127)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand
ler.java:515)
at
org.eclipse.jetty.server.handler.ContextHandler.
doScope(ContextHandler.java:1061)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(
ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(
HandlerWrapper.java:97)
at org.eclipse.jetty.server.Serve
r.handle(Server.java:499)
at org.eclipse.jetty.server.HttpC
hannel.handle(HttpChannel.java:311)
at
org.eclipse.jetty.server.HttpConnection.onFillable(
HttpConnection.java:257)
at
org.eclipse.jetty.io <http://org.eclipse.jetty.io>.
AbstractConnection$2.run(AbstractConnection.java:544)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
My configuration:
<route id="ds-rest-archive-upload-processor">
<from uri="jetty:
https://storage:8443/data/archive?sslContextParametersRef= <
https://storage:8443/data/archive?sslContextParametersRef=>
sslContextParameters&requestBufferSize=32768&responseBufferS
ize=32768&
httpMethodRestrict=POST
<https://storage:8443/data/archive?sslContextParametersRef=
<https://storage:8443/data/archive?sslContextParametersRef=>
sslContextParameters&requestBufferSize=32768&responseBufferS
ize=32768&
httpMethodRestrict=POST>"
/>
<to uri="bean:uploadFromArchiveFileProcessor?method=process"
pattern="InOut" />
</route>
<bean id="jetty"
class="org.apache.camel.component.jetty9.JettyHttpComponent9
">
<property name="endpointClass"
value="org.apache.camel.component.jetty9.JettyHttpEndpoint9"
/>
<property name="sslContextParameters"
ref="sslContextParameters" />
<property name="useContinuation" value="true" />
<property name="requestBufferSize" value="32768" />
<property name="responseBufferSize" value="32768" />
<property name="threadPool">
<bean class="org.eclipse.jetty.util.
thread.QueuedThreadPool">
<constructor-arg index="0" value="1000" />
</bean>
</property>
<property name="jettyHttpBinding">
<bean class="org.apache.camel.compon
ent.jetty.DefaultJettyHttpBinding">
<property name="transferException" value="false" />
</bean>
</property>
</bean>
My camel application is running in docker container with
limited amount of
memory, and I did not expect, that a file will be read a
string instead of
reading from InputStream.
Is there any way to consume large files with small amount of
memory?
Regards,
Uladzimir
