I am working with a route which downloads files from a remote server to local directory using SFTP.
Apache Camel version: 2.15.2 >From endpoint: sftp://xxx.xxx.xxx.xxx:xx//User/User01?delay=30s&include=File.*.csv&initialDelay=1m&password=xxxxxx&stepwise=false&streamDownload=true&username=User01 To endpoint: file:///var/opt/myfolder/incoming?doneFileName=${file:name}.done The remote location has more than 10 files available for download. After downloading 2-3 files, the route gets stuck and after around 30 seconds I get the below error in logs: DEBUG 04/11/15 07:45:24,183 org.apache.camel.component.file.FileOperations :Using InputStream to write file: /var/opt/myfolder/incoming/File01.csv ... around 30 secs gap ... INFO 04/11/15 07:49:53,820 org.apache.camel.component.file.remote.SftpOperations$JSchLogger :JSCH -> Caught an exception, leaving main loop due to Connection reset INFO 04/11/15 07:49:53,821 org.apache.camel.component.file.remote.SftpOperations$JSchLogger :JSCH -> Disconnecting from xxx.xxx.xxx.xxx port xx WARN 04/11/15 07:49:53,823 org.apache.camel.util.IOHelper :Cannot close: File01.csv. Reason: Pipe closed java.io.IOException: Pipe closed at java.io.PipedInputStream.read(PipedInputStream.java:308) at java.io.PipedInputStream.read(PipedInputStream.java:378) at java.io.InputStream.skip(InputStream.java:222) at com.jcraft.jsch.ChannelSftp.skip(ChannelSftp.java:2894) at com.jcraft.jsch.ChannelSftp.access$600(ChannelSftp.java:36) at com.jcraft.jsch.ChannelSftp$RequestQueue.cancel(ChannelSftp.java:1246) at com.jcraft.jsch.ChannelSftp$2.close(ChannelSftp.java:1503) at org.apache.camel.util.IOHelper.close(IOHelper.java:326) at org.apache.camel.component.file.FileOperations.writeFileByStream(FileOperations.java:404) at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:274) at org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:277) at org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:165) at org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:79) at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) at org.apache.camel.processor.Pipeline.process(Pipeline.java:118) at org.apache.camel.processor.Pipeline.process(Pipeline.java:80) at org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:111) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) at org.apache.camel.processor.Pipeline.process(Pipeline.java:118) at org.apache.camel.processor.Pipeline.process(Pipeline.java:80) at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109) at org.apache.camel.processor.Pipeline.process(Pipeline.java:60) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:166) at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:435) at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137) at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211) at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175) at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174) at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) I have set redelivery so it re attempts the download. The reattempt happens for the last file and as per the logs the file is downloaded. But when i check the folder the file size is 0 and even the .done file is created, the actual file size on remote server is 28 KB. For rest of the files i get the below error for each file and none of the file is downloaded: WARN 04/11/15 07:49:58,877 org.slf4j.helpers.MarkerIgnoringBase :Error processing file RemoteFile[/User/User01/File02.csv] due to Cannot retrieve file: /User/User01/File02.csv. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot retrieve file: /User/User01/File02.csv] org.apache.camel.component.file.GenericFileOperationFailedException: Cannot retrieve file: /User/User01/File02.csv at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:651) at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:594) at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:396) at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137) at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211) at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175) at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174) at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: 4: at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1513) at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1266) at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:636) ... 14 more Caused by: java.io.IOException: Pipe closed at java.io.PipedInputStream.read(PipedInputStream.java:308) at com.jcraft.jsch.Channel$MyPipedInputStream.updateReadSide(Channel.java:362) at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1287) ... 16 more I tried using both disconnect=true and false, the issue is happening in both the cases. Any suggestions what could be wrong? Thanks, Sumeet -- View this message in context: http://camel.465427.n5.nabble.com/SFTP-endpoint-Pipe-closed-issue-tp5773371.html Sent from the Camel - Users mailing list archive at Nabble.com.
