If your service runs fine with a smaller file, and you are running into trouble with a particular file size, that means you are running into memory issues. The reason why you are experiencing a 2x expected run-time for that particular test is because of all the memory garbage collections that the VM is supposed to do.
My guess is that you are passing in the file as an byte array or similar. This, unless you have a need to do so, is probably not a good idea. Pass it in as an input stream, and write it out to your particular storage device. (One of my services directly reads from the input stream and writes to a file-based output stream.) That way, your memory requirements are just the buffers for the stream and you should be able to do whatever sized file you need. CXF does not have a file limitation. It's limited by your allocated memory and the way that you map input types. Jeff On Sun, Jan 22, 2012 at 4:21 PM, noosy <[email protected]> wrote: > Hi, > > I have a CXF JAX-RS upload service which consumes("multipart/form-data") and > accepts a MultipartBody object as a param. > > I have an integration test which tries to see how big a file I can upload to > this service. I am able to upload a file of size 321 MB successfully. This > is the largest file which passes for me. The issue is, intermittently when I > run this test, the test passes, but it takes ages to run (twice as long as > usual) and I see these warnings in the console: > > 20-Jan-2012 06:47:37 WARNING: Interceptor for > {http://my.company/}UploadService has thrown exception, unwinding now > 20-Jan-2012 06:47:37 org.apache.cxf.interceptor.Fault: Could not send > Message. > 20-Jan-2012 06:47:37 at > org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:64) > 20-Jan-2012 06:47:37 at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263) > 20-Jan-2012 06:47:37 at > org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77) > 20-Jan-2012 06:47:37 at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:323) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:289) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:72) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:942) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:878) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.Server.handle(Server.java:345) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:441) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:936) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:801) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:224) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:52) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533) > 20-Jan-2012 06:47:37 at java.lang.Thread.run(Thread.java:662) > 20-Jan-2012 06:47:37 Caused by: org.eclipse.jetty.io.EofException > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:919) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpGenerator.complete(HttpGenerator.java:812) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpConnection.commitResponse(HttpConnection.java:572) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpConnection$Output.close(HttpConnection.java:992) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http.AbstractHTTPDestination$WrappedOutputStream.close(AbstractHTTPDestination.java:650) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:593) > 20-Jan-2012 06:47:37 at > org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62) > 20-Jan-2012 06:47:37 ... 23 more > 20-Jan-2012 06:47:37 Caused by: java.nio.channels.ClosedChannelException > 20-Jan-2012 06:47:37 at > sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:135) > 20-Jan-2012 06:47:37 at > sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:357) > 20-Jan-2012 06:47:37 at > java.nio.channels.SocketChannel.write(SocketChannel.java:360) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.ChannelEndPoint.gatheringFlush(ChannelEndPoint.java:346) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:284) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:300) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:846) > 20-Jan-2012 06:47:37 ... 30 more > 20-Jan-2012 06:47:37 Jan 20, 2012 6:47:37 AM > org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging > 20-Jan-2012 06:47:37 WARNING: Interceptor for > {http://my.company/}UploadService has thrown exception, unwinding now > 20-Jan-2012 06:47:37 org.apache.cxf.interceptor.Fault: XML_WRITE_EXC > 20-Jan-2012 06:47:37 at > org.apache.cxf.binding.xml.interceptor.XMLFaultOutInterceptor.handleMessage(XMLFaultOutInterceptor.java:87) > 20-Jan-2012 06:47:37 at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263) > 20-Jan-2012 06:47:37 at > org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:107) > 20-Jan-2012 06:47:37 at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:323) > 20-Jan-2012 06:47:37 at > org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77) > 20-Jan-2012 06:47:37 at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:263) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:323) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:289) > 20-Jan-2012 06:47:37 at > org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:72) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:942) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:878) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.Server.handle(Server.java:345) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:441) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:936) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:801) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:224) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:52) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533) > 20-Jan-2012 06:47:37 at java.lang.Thread.run(Thread.java:662) > 20-Jan-2012 06:47:37 Caused by: com.ctc.wstx.exc.WstxIOException: null > 20-Jan-2012 06:47:37 at > com.ctc.wstx.sw.BaseStreamWriter.flush(BaseStreamWriter.java:257) > 20-Jan-2012 06:47:37 at > org.apache.cxf.binding.xml.interceptor.XMLFaultOutInterceptor.handleMessage(XMLFaultOutInterceptor.java:85) > 20-Jan-2012 06:47:37 ... 25 more > 20-Jan-2012 06:47:37 Caused by: org.eclipse.jetty.io.EofException > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:919) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:452) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:94) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpConnection$Output.flush(HttpConnection.java:1009) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:173) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:101) > 20-Jan-2012 06:47:37 at > org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46) > 20-Jan-2012 06:47:37 at > com.ctc.wstx.sw.EncodingXmlWriter.flushBuffer(EncodingXmlWriter.java:697) > 20-Jan-2012 06:47:37 at > com.ctc.wstx.sw.EncodingXmlWriter.flush(EncodingXmlWriter.java:171) > 20-Jan-2012 06:47:37 at > com.ctc.wstx.sw.BaseStreamWriter.flush(BaseStreamWriter.java:255) > 20-Jan-2012 06:47:37 ... 26 more > 20-Jan-2012 06:47:37 Caused by: java.nio.channels.ClosedChannelException > 20-Jan-2012 06:47:37 at > sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:135) > 20-Jan-2012 06:47:37 at > sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:357) > 20-Jan-2012 06:47:37 at > java.nio.channels.SocketChannel.write(SocketChannel.java:360) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.ChannelEndPoint.gatheringFlush(ChannelEndPoint.java:346) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:284) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:300) > 20-Jan-2012 06:47:37 at > org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:846) > 20-Jan-2012 06:47:37 ... 35 more > > This concerns me and it means the build (runs unit and integration tests) > takes much longer than it should to run. > > A couple of things. > > I am using CXF 2.5.1. > > I am using AbstractBusTestServerBase to do my testing (standing up a test > server). > > I added the following to my spring context file but I'm not sure if it is > even doing anything. I cannot see the attachment directory being created > (where should I be able to find it assuming it is not being automatically > cleaned up): > > <jaxrs:server id="upload-service" address="/upload-service"> > <jaxrs:serviceBeans> > <ref bean="uploadService"/> > </jaxrs:serviceBeans> > <jaxrs:properties> > <entry key="attachment-directory" value="/temp/attachments"/> > <entry key="attachment-memory-threshold" value="4000000"/> > <entry key="attachment-max-size" value="409600000"/> > </jaxrs:properties> > </jaxrs:server> > > How big a file should I be able to upload before tweaking app server > settings (assuming this could boost it further)? > > Appreciate any pointers. > > Thanks. > > -- > View this message in context: > http://cxf.547215.n5.nabble.com/Issue-uploading-large-attachment-to-JAX-RS-service-tp5165047p5165047.html > Sent from the cxf-user mailing list archive at Nabble.com.
