Thanks Mark. This is really helpful. Will try this option. Sent from my iPhone
> On Apr 29, 2020, at 1:00 AM, Mark Payne <[email protected]> wrote: > > Asmath, > > NiFi is not job oriented, and it doesn’t really have any notion of being > “done” - it monitors the directory continuously, as you noted. > > What might be of use to you is the MonitorActivity processor and the > Wait/Notify Processors. I recommend that you read up on their documentation. > But you could potentially do something along the lines of: > > > ListFile — success —> MonitorActivity — success —> FetchFile —> Wait > — inactive —> > Notify > > So it should pull the data in, and then wait. > The MonitorActivity will detect that no more data has come in for some amount > of time (say 5 minutes) and as a result determine that the “job is done” and > send a FlowFile to Notify to unblock the waiting FlowFiles. > > Hope this helps > -Mark > > >> On Apr 28, 2020, at 3:07 AM, KhajaAsmath Mohammed <[email protected]> >> wrote: >> >> Hi, >> >> We are using List file and Fetch file to continuously monitor shared >> location to read the files and process the next level of jobs. Issue we are >> facing is to wait for all the files to be read before running the next jobs. >> >> Currently list file has 5 files but during the fetch process , it reads file >> one by one. Next processor after fetch will work on the files but we have to >> wait until all the files are read before the next processor runs. How to >> acheive this. >> >> Number of files keeps changing. >> >> Thanks, >> Asmath >
