from(" sftp://X.X.X.X:2222/ts/sdf?username=sftpuser&password=*****&delay=100&initialDelay=100&streamDownload=false&recursive=false&readLock=changed&disconnect=true&KnownHostsFile=~/.ssh/known_hosts&localWorkDirectory=/tmp&moveFailed=/ts/error&include=.*\.xml&move=inprogress") .routePolicy(new ZooKeeperRoutePolicy("{{zookeeper." + feedName + ".forwarder.endpoint}}", 1)) .to("sftp://Y.Y.Y.T:2222/sftp/sdf?username=sftpuser&password=*****&fileExist=move&moveExisting=20170112221528-***.xml&KnownHostsFile=~/.ssh/known_hosts");
I narrow done the issue to above code snippet, its FTPing the file from 1 FTP server to another with move=inprogress folder using the ZooKeeperRoutePolicy. - it works as expected on the single instance. - on having multiple instance(of camel route) , FTPing is successful but move file to inprogress is failing due to below error. However this error happen only on the first file(205.xml). As you can see on the log, the subsequence file(206.xml) are transfer sucessfully as expected according to error message it is "Caused by: com.jcraft.jsch.SftpException: No such file." But if i remove the "move=inprogress" from the route, i can see the file in /sftp/sd folder. is there any other way to FTP the file and move the file to folder(using zookeeper for leader election)? or how do i eliminate this error? 2017-01-12T00:11:31.768517747Z 2017-01-12 00:11:31 INFO [vzw/sdf] [o.a.c.c.z.policy.ZooKeeperElection :138] Initializing ZookeeperElection with uri '{{zookeeper.sdf.forwarder.endpoint}}' 2017-01-12T00:11:31.788697843Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT 2017-01-12T00:11:31.788763764Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:host.name=a810a44c767c 2017-01-12T00:11:31.788774252Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:java.version=1.8.0_60 2017-01-12T00:11:31.788783141Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:java.vendor=Oracle Corporation 2017-01-12T00:11:31.788791331Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:java.home=/opt/jre1.8.0_60 2017-01-12T00:11:31.789350564Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:java.class.path=/app.jar 2017-01-12T00:11:31.789388119Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-01-12T00:11:31.790108758Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:java.io.tmpdir=/tmp 2017-01-12T00:11:31.791108364Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:java.compiler=<NA> 2017-01-12T00:11:31.791583396Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:os.name=Linux 2017-01-12T00:11:31.792170295Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:os.arch=amd64 2017-01-12T00:11:31.792194966Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:os.version=4.4.39-moby 2017-01-12T00:11:31.792379046Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:user.name=root 2017-01-12T00:11:31.793107176Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:user.home=/root 2017-01-12T00:11:31.793138639Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :100] Client environment:user.dir=/etc/service/dataf_gateway 2017-01-12T00:11:31.794761800Z 2017-01-12 00:11:31 INFO [vzw/sdf] [org.apache.zookeeper.ZooKeeper :438] Initiating client connection, connectString=68.128.182.149:2181 sessionTimeout=5000 watcher=org.apache.camel.component.zookeeper.ConnectionHolder@4ec951ba 2017-01-12T00:11:31.813834319Z 2017-01-12 00:11:31 INFO [9:2181)] [org.apache.zookeeper.ClientCnxn :032] Opening socket connection to server 68.128.182.149/68.128.182.149:2181. Will not attempt to authenticate using SASL (unknown error) 2017-01-12T00:11:31.904467961Z 2017-01-12 00:11:31 INFO [9:2181)] [org.apache.zookeeper.ClientCnxn :876] Socket connection established to 68.128.182.149/68.128.182.149:2181, initiating session 2017-01-12T00:11:32.006541339Z 2017-01-12 00:11:32 INFO [9:2181)] [org.apache.zookeeper.ClientCnxn :299] Session establishment complete on server 68.128.182.149/68.128.182.149:2181, sessionid = 0x159855fc87400c2, negotiated timeout = 5000 2017-01-12T00:11:32.109418356Z 2017-01-12 00:11:32 WARN [vzw/sdf] [o.a.c.c.zookeeper.ZookeeperProducer:237] Node '/zk/file_forwarder_node/a810a44c767c-8a3d4433-5042-4e24-bf45-694afc61c66a' did not exist, creating it. 2017-01-12T00:11:32.398124509Z 2017-01-12 00:11:32 INFO [vzw/sdf] [o.a.c.c.z.policy.ZooKeeperElection :151] Candidate node '/zk/file_forwarder_node/a810a44c767c-8a3d4433-5042-4e24-bf45-694afc61c66a' has been created 2017-01-12T00:11:32.431922439Z 2017-01-12 00:11:32 INFO [vzw/sdf] [o.a.camel.impl.DefaultCamelContext :570] Route: election-route-a810a44c767c-8a3d4433-5042-4e24-bf45-694afc61c66a started and consuming from: Endpoint[zookeeper://68.128.182.149:2181/zk/file_forwarder_node?createMode=EPHEMERAL] 2017-01-12T00:11:32.620071603Z 2017-01-12 00:11:32 INFO [vzw/sdf] [Datafeed.Sdf.dc3Forwarder :159] Moving Starts fileName=205.xml from /ts/vzw/sdf 2017-01-12T00:11:39.674491004Z 2017-01-12 00:11:39 INFO [vzw/sdf] [Datafeed.Sdf.dc3Forwarder :159] Moving Completed fileName=205.xml transfer to sftp://68.128.182.149:2222 2017-01-12T00:11:40.692277702Z 2017-01-12 00:11:40 WARN [vzw/sdf] +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [o.a.c.c.f.GenericFileOnCompletion :213] Error during commit. Exchange[ID-a810a44c767c-44403-1484179857573-0-2]. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot rename file from: ts/sdf/205.xml to: ts/sdf/inprogress/205.xml] 2017-01-12T00:11:40.692309664Z org.apache.camel.component.file.GenericFileOperationFailedException: Cannot rename file from: ts//sdf/205.xml to: ts/sdf/inprogress/205.xml 2017-01-12T00:11:40.692319752Z at org.apache.camel.component.file.remote.SftpOperations.renameFile(SftpOperations.java:431) 2017-01-12T00:11:40.692331238Z at org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:113) 2017-01-12T00:11:40.692336731Z at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:88) 2017-01-12T00:11:40.692341626Z at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:127) 2017-01-12T00:11:40.692349716Z at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83) 2017-01-12T00:11:40.692354510Z at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:57) 2017-01-12T00:11:40.692359205Z at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:104) 2017-01-12T00:11:40.692364099Z at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:229) 2017-01-12T00:11:40.692375485Z at org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:65) 2017-01-12T00:11:40.692381378Z at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:674) 2017-01-12T00:11:40.692386372Z at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:629) 2017-01-12T00:11:40.692390967Z at org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:246) 2017-01-12T00:11:40.692395761Z at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:179) 2017-01-12T00:11:40.692409245Z at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:454) 2017-01-12T00:11:40.692414039Z at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137) 2017-01-12T00:11:40.692418534Z at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:226) 2017-01-12T00:11:40.692422929Z at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:190) 2017-01-12T00:11:40.692427323Z at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175) 2017-01-12T00:11:40.692432817Z at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102) 2017-01-12T00:11:40.692437211Z at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 2017-01-12T00:11:40.692441606Z at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 2017-01-12T00:11:40.692449497Z at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) 2017-01-12T00:11:40.692457887Z at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) 2017-01-12T00:11:40.692464779Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2017-01-12T00:11:40.692469273Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2017-01-12T00:11:40.692474567Z at java.lang.Thread.run(Thread.java:745) 2017-01-12T00:11:40.692479761Z Caused by: com.jcraft.jsch.SftpException: No such file 2017-01-12T00:11:40.692484055Z at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2846) 2017-01-12T00:11:40.692488550Z at com.jcraft.jsch.ChannelSftp.rename(ChannelSftp.java:1923) 2017-01-12T00:11:40.692493245Z at org.apache.camel.component.file.remote.SftpOperations.renameFile(SftpOperations.java:428) 2017-01-12T00:11:40.692503133Z ... 25 common frames omitted 2017-01-12T00:12:08.133521922Z 2017-01-12 00:12:08 INFO [/sdf] [Dataf.Sdf.dc3Forwarder :159] Moving Starts fileName=206.xml from /ts/sdf 2017-01-12T00:12:11.826120872Z 2017-01-12 00:12:11 INFO [/sdf] [Dataf.Sdf.dc3Forwarder :159] Moving Completed fileName=206.xml transfer to sftp://X.X.X.X:2222 -- View this message in context: http://camel.465427.n5.nabble.com/SFTP-leader-election-using-ZooKeeper-RoutePolicy-Error-on-moving-file-tp5792480.html Sent from the Camel - Users mailing list archive at Nabble.com.