Hello, It sounds like you would need a modified version of ListSFTP that supported incoming flow files, and also supported storing state for more than one hostname/directory.
It might be possible to do something ExecuteStreamCommand... you could get a config file with GetFile, send it to SplitText, and then send each hostname to ExecuteStreamCommand which could perform SFTP commands, just not sure if you can simulate the List+Fetch behavior that the actual processors give you. Other options... you might be able to script the creation of your ListSFTP processors using the NiFi REST API, or if you have access to the source systems you could also consider running a MiNiFi agent on all the source systems and sending the data back to a central NiFi via site-to-site. Thanks, Bryan On Tue, Oct 4, 2016 at 10:46 AM, Provenzano Nicolas < [email protected]> wrote: > Hi all, > > > > We need to collect files using SFTP from several data sources. > > > > We can’t add one ListSFTP processor per data source since, first, it would > require define many and it’s not dynamic at all. > > > > Our initial idea was to add processors to read a configuration file > (containing a list of hostnames) and create one flow file per hostname. > > > > However, the ListSFTP does not accept input connections so it can’t use > the output of the previous processors. > > > > Is there any way of doing it using native processors or do we need to > define our own processor ? > > > > Thanks in advance > > > > BR > > > > Nicolas >
