Hi A bit odd as the output stream passed in to the ftp client is a FileOutputStream. I suggest if you can check the commons-net code and see what it does.
There was a bug though if you afterwards route to a file endpoint. Then Camel was optimized to try first to rename the file from localWorkDir, than doing a file copy. This bug has been fixed and will be in next releases. But this is not your problem, as its when you download the file, it goes into memory. Which it ought to stream to the file straight. Maybe with a little in memory buffer (but that would be commons-net specific). On Thu, Oct 25, 2012 at 4:51 PM, Achim Nierbeck <[email protected]> wrote: > Hi, > > using Camel 2.10.2 > I have a ftp endpoint that downloads a couple of files from a remote > FTP server. > There are different files located on this server, some of them are > about 700 MB size. > Now every time I start the route for downloading it gives me a OOM > when trying to download this file. > The route is configured to use the *localWorkDirectory*. > Roughly the route does something like the following: > > from("ftp://server?noop=true&localWorkDirectory=/myTemp&sendEmptyMessageWhenIdle=true";) > .onException() > ... ommited the exception part .... > .onCompletion() > ... omitted the onCompletion part .... > .choice() > .when(body().isNull()) > .... stop the whole thing here ... > .otherwise() > .to("file:/dropItHere") > .setBody(constant("")) > .to("seda:doSomeLogging"); > > I tried to increase the memory up to 3GB for the Heap size with no luck. > > Following Stacktrace is logged: > Camel (rootContext) thread #28 - ftp://usrXXX@ServerXXXX/ > at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:25) > at java.util.Arrays.copyOf([BI)[B (Arrays.java:2786) > at java.io.ByteArrayOutputStream.write([BII)V > (ByteArrayOutputStream.java:94) > at > org.apache.commons.net.io.Util.copyStream(Ljava/io/InputStream;Ljava/io/OutputStream;IJLorg/apache/commons/net/io/CopyStreamListener;Z)J > (Util.java:123) > at > org.apache.commons.net.ftp.FTPClient._retrieveFile(Ljava/lang/String;Ljava/lang/String;Ljava/io/OutputStream;)Z > (FTPClient.java:1695) > at > org.apache.commons.net.ftp.FTPClient.retrieveFile(Ljava/lang/String;Ljava/io/OutputStream;)Z > (FTPClient.java:1669) > at > org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(Ljava/lang/String;Lorg/apache/camel/Exchange;)Z > (FtpOperations.java:328) > at > org.apache.camel.component.file.remote.FtpOperations.retrieveFile(Ljava/lang/String;Lorg/apache/camel/Exchange;)Z > (FtpOperations.java:297) > at > org.apache.camel.component.file.GenericFileConsumer.processExchange(Lorg/apache/camel/Exchange;)V > (GenericFileConsumer.java:317) > at > org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(Lorg/apache/camel/Exchange;)V > (RemoteFileConsumer.java:94) > at > org.apache.camel.component.file.GenericFileConsumer.processBatch(Ljava/util/Queue;)I > (GenericFileConsumer.java:189) > at org.apache.camel.component.file.GenericFileConsumer.poll()I > (GenericFileConsumer.java:155) > at org.apache.camel.impl.ScheduledPollConsumer.doRun()V > (ScheduledPollConsumer.java:139) > at org.apache.camel.impl.ScheduledPollConsumer.run()V > (ScheduledPollConsumer.java:91) > > > The attached OOM-Camel.PNG shows what the Memory Analysis tells me. > > Am I missing something critical here? Or do I have to tune up the > memory for those kind > of cases? > > > Thanks and regards, Achim > -- > > Apache Karaf <http://karaf.apache.org/> Committer & PMC > OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> > Committer & Project Lead > OPS4J Pax for Vaadin > <http://team.ops4j.org/wiki/display/PAXVAADIN/Home> Commiter & Project > Lead > blog <http://notizblog.nierbeck.de/> -- Claus Ibsen ----------------- Red Hat, Inc. FuseSource is now part of Red Hat Email: [email protected] Web: http://fusesource.com Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen
