Dave, The next version of NiFi has a FetchSFTP processor which may simplify your design. E.g. I have it receiving file paths to pull via many incoming channels, which can be different directories on a server (haven't looked, but it might support server as an expression field).
Andrew On 10/6/15, 4:19 PM, "Joe Witt" <[email protected]> wrote: >David, > >You can already place multiple GetSFTP processors on a single flow to >do what is described here. Capturing of *many* flows using one or >more protocols at once and routing them is quite common. > >Thanks >Joe > >On Tue, Oct 6, 2015 at 3:58 PM, DAVID SMITH <[email protected]> wrote: >> Hi >> >> Has anyone or is anyone creating a bulk data ingest processor. We are >> looking into the feasibility of using NiFi to do our bulk data pickups from >> many locations and possibly multiple directories from each location. >> My initial thoughts were to use a processor which could run multiple SFTP >> sessions at one time. Can anyone give me any guidance and what pitfalls I >> may come up against? >> >> Many thanks >> Dave >> >> Sent from Yahoo! Mail on Android >> >
