I took the FromFtpMoveFileTest and modified it for SFTP - that works, so "move" works for SFTP. My problem is that because the current release requires hard-coding the URI for (s)ftp(s) consumer - I implemented a custom processor that instantiates the SFTP component, SFTP endpoint and SFTP consumer, all programatically. Obviously, I'm missing something. Unfortunately, it's pretty much a show-stopper for us to not be able to SFTP *from* (consume) dynamically (runtime-calculated) URIs.
On Wed, Apr 17, 2013 at 4:54 PM, Chris Wolf <cwolf.a...@gmail.com> wrote: > I notice there's a unit test, FromFtpMoveFileTest, for plain FTP, but > not for SFTP - should I file a bug, or am I doing something > wrong? > > On Wed, Apr 17, 2013 at 11:39 AM, Chris Wolf <cwolf.a...@gmail.com> wrote: >> Trying to use Sftp component. I am using localWorkDirectory because >> I don't want the whole files loaded into memory. >> Seems like the files *are* being downloaded but they are NOT being >> moved, on the remote side to the directory indicated by the "move" >> option. >> >> >> Here, on a poll, it gets a file, while downloading to >> "localWorkDirectory", the file's name is appended with ".inprogress", >> then it's >> renamed to the base file name without ".inprogress" - so far, so good, >> except on the server-side, it's not moved to the "move" directory. >> >> [#4 - sftp://localhost/download] SftpOperations DEBUG Retrieve file to >> local work file result: true >> [#4 - sftp://localhost/download] SftpOperations TRACE Renaming local >> in progress file from: >> \tmp\local\ftpwork\ADPT5_MKT_20120319_VOIR_DAILY.csv.inprogress to: >> \tmp\local\ftpwork\ADPT5_MKT_20120319_VOIR_DAILY.csv >> [#4 - sftp://localhost/download] SftpConsumer TRACE Retrieved file: >> download/ADPT5_MKT_20120319_VOIR_DAILY.csv from: >> Endpoint[sftp://localhost/download] >> >> ...then, at poll n+1, I see: >> >> [#4 - sftp://localhost/download] SftpConsumer TRACE >> SftpFile[fileName=ADPT5_MKT_20120319_VOIR_DAILY.csv, >> longName=-rwxrwxrwx 1 adpt5 adpt5 298313 Apr 17 09:42 >> ADPT5_MKT_20120319_VOIR_DAILY.csv, dir=false] >> [#4 - sftp://localhost/download] SftpConsumer TRACE Skipping as file >> is already in progress: ADPT5_MKT_20120319_VOIR_DAILY.csv >> >> I guess that's the idempotent mechanism, but that file should have >> been moved out of that directory. >> >> >> SftpConfiguration: >> >> chmod = null, >> ciphers = null, >> compression = 0, >> connectTimeout = 10000, >> directory = download, >> host = localhost, >> knownHostsFile = null, >> password = adpt5, >> port = 22, >> privateKeyFile = null, >> privateKeyFilePassphrase = null, >> protocol = sftp, >> separator = UNIX, >> serverAliveCountMax = 1, >> serverAliveInterval = 0, >> siteCommand = null, >> soTimeout = 0, >> strictHostKeyChecking = no, >> timeout = 30000, >> username = adpt5, >> binary = true, >> passiveMode = false, >> stepwise = true, >> throwExceptionOnConnectFailed = false >> >> [1) thread #0 - timer://kickoff] INFO SftpEndpoint: >> antFilter = null, >> bufferSize = 131072, >> camelContext = CamelContext(camel-1), >> charset = null, >> component = org.apache.camel.component.file.remote.SftpComponent@1e34f445, >> configuration = download, >> consumerProperties = null, >> doneFileName = null, >> endpointConfiguration = >> org.apache.camel.impl.MappedEndpointConfiguration@8292f75a, >> endpointKey = sftp://localhost/download, >> endpointUri = sftp://localhost/download, >> exchangePattern = InOnly, >> exchanges = [Exchange[ADPT5_MKT_20090211_MICF.csv], >> Exchange[ADPT5_MKT_20090211_MINOCF.csv], >> Exchange[ADPT5_MKT_20090305_VOE.csv], >> Exchange[ADPT5_MKT_20090305_VOIR.csv], >> Exchange[ADPT5_MKT_20120319_DAILY.csv], >> Exchange[ADPT5_MKT_20120319_FX.csv], >> Exchange[ADPT5_MKT_20120319_MTX_DAILY.csv], >> Exchange[ADPT5_MKT_20120319_VOIR_DAILY.csv], >> Exchange[ADPT5_MKT_20121121_HISTORY_EUR_IRBank-FZ-D.csv], >> Exchange[ADPT5_MKT_20121121_HISTORY_EUR_IRBank-FZ-M.csv], >> Exchange[ADPT5_MKT_20121121_HISTORY_EUR_IRBank-FZ-Y.csv]], >> exclude = null, >> exclusiveReadLockStrategy = null, >> fileExist = Override, >> fileName = null, >> fileSeparator = /, >> filter = ms.algo.adapt.camel.cpmd.CPMDFileFilter@54aa2db, >> genericFileProcessStrategy = >> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy@67b14530, >> id = endpoint13, >> idempotentRepository = null, >> include = null, >> inProgressRepository = >> org.apache.camel.processor.idempotent.MemoryIdempotentRepository@563b100c, >> localWorkDirectory = /tmp/local/ftpwork, >> maxDepth = 2147483647, >> maximumReconnectAttempts = 3, >> maxMessagesPerPoll = 0, >> minDepth = 0, >> move = ${file:parent}/../archive/${file:onlyname},, >> moveExisting = null, >> moveFailed = null, >> preMove = null, >> processStrategy = >> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy@67b14530, >> readLock = none, >> readLockCheckInterval = 5000, >> readLockMinLength = 1, >> readLockTimeout = 20000, >> reconnectDelay = 1000, >> scheme = sftp, >> sortBy = null, >> sorter = null, >> status = Started, >> tempFileName = null, >> tempPrefix = null, >> version = 2.10.4, >> allowNullBody = false, >> autoCreate = true, >> delete = false, >> directoryMustExist = false, >> disconnect = false, >> eagerDeleteTargetFile = true, >> eagerMaxMessagesPerPoll = true, >> fastExistsCheck = false, >> flatten = false, >> keepLastModified = false, >> lenientProperties = false, >> noop = false, >> recursive = false, >> runAllowed = true, >> singleton = true, >> started = true, >> starting = false, >> startingDirectoryMustExist = false, >> stopped = false, >> stopping = false, >> suspended = false, >> suspending = false, >> synchronous = false
