Hi Mohit, ListFile expects the directory to be on both nodes because the Primary node could swap at some point, and so the configuration has to be valid for the whole cluster. If you want to have a single directory which ListFile/FetchFile pulls from, you could try a cluster filesystem or an SFTP service as the source perhaps? Using a directory on the local file system is probably not going to be a robust solution.
On Fri, Feb 23, 2018 at 8:42 AM <mohit.j...@open-insights.co.in> wrote: > > > I tried it the same way, but ListFile expects the directory location to be > present in both the nodes. > I found way to create a List/Fetch processor on standalone nifi and > connect it to the cluster using Remote Process Group. But I’m looking for > any other approach which doesnot involve Remote Process Group/ Standalone > Nifi. > > > > > > *From:* Marco Gaido [mailto:marcogaid...@gmail.com] > *Sent:* 23 February 2018 14:08 > *To:* users@nifi.apache.org > *Subject:* Re: How to use List File/Fetch File processor in a cluster > mode. > > > > Hi Mohit, > > > > I'd suggest you to schedule it only on the primary node instead of on all > nodes and after that repartition the outcoming flow files among all the > available nodes. > > > > Best regards, > > Marco > > > > 2018-02-23 6:34 GMT+01:00 <mohit.j...@open-insights.co.in>: > > Hi all, > > > > I created a template to ingest data from any NiFi local directory using > ListFile/FetchFile processor. When running in a cluster mode it asks for > the directory in both the nodes. What is the best way to use the > combination of both? > > > > > > Regards, > > Mohit Jain > > >
