Claus, is the ticket you're referring to CAMEL-6058: camel-ftp - Allow concurrent download of files? The verbiage in that ticket appears to refer only to the FTP consumer, whereas my use case requires the SFTP consumer.
Additionally, I've learned more about the network I'm working with and found that download speeds are throttled per connection, not per download request. My current line of thought is that an SFTP consumer connection pool may be the solution I'm looking for. Though perhaps a connection pool would be implicit in any realization of concurrent downloads, as the JIRA ticket description suggests for the FTP consumer. Is CAMEL-6058 still the appropriate JIRA ticket for my use case requiring the SFTP consumer? On Tue, Jun 26, 2018 at 6:05 AM, Claus Ibsen <[email protected]> wrote: > Hi > > There is a JIRA ticket about concurrent downloads. Use the vote in > JIRA to tell us that you like that. > > > > On Tue, Jun 19, 2018 at 7:30 PM, Alexander Barrow <[email protected]> wrote: >> I am developing an application that downloads large files from a >> remote SFTP server over a bandwidth limited network. Download speeds >> are bottle-necked, making it highly desirable to be able to download >> multiple files concurrently. From what I've gathered, this is not a >> feature supported out of the box by the standard SFTP component. >> >> A solution I engineered to work around this was to use a route defined >> something like as follows: >> >> from( "sftp:user@host?download=false..." ) >> .threads( 5 ).maxQueueSize( 0 ) >> .pollEnrich().simple( >> "sftp:user@host?localWorkDirectory=.inbound&include=$simple{file:name}&disconnect=true..." >> ) >> .to( "file:downloads" ); >> >> This uses the route's consumer to supply a "list" of files to >> download, but delegates the actual downloading to the poll-enrich EIP >> across multiple threads. The issue with this approach is that it uses >> multiple connections and repeatedly connects and disconnects from the >> remote server, which the SFTP server owners dislike. >> >> Is there another existing component or feature that would support the >> multiple concurrent downloads use case, or is this a gap in Camel's >> feature list? > > > > -- > Claus Ibsen > ----------------- > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
