Asmath,
NiFi is not job oriented, and it doesn’t really have any notion of being “done”
- it monitors the directory continuously, as you noted.
What might be of use to you is the MonitorActivity processor and the
Wait/Notify Processors. I recommend that you read up on their documentation.
But you could potentially do something along the lines of:
ListFile — success —> MonitorActivity — success —> FetchFile —> Wait
— inactive —> Notify
So it should pull the data in, and then wait.
The MonitorActivity will detect that no more data has come in for some amount
of time (say 5 minutes) and as a result determine that the “job is done” and
send a FlowFile to Notify to unblock the waiting FlowFiles.
Hope this helps
-Mark
> On Apr 28, 2020, at 3:07 AM, KhajaAsmath Mohammed <[email protected]>
> wrote:
>
> Hi,
>
> We are using List file and Fetch file to continuously monitor shared location
> to read the files and process the next level of jobs. Issue we are facing is
> to wait for all the files to be read before running the next jobs.
>
> Currently list file has 5 files but during the fetch process , it reads file
> one by one. Next processor after fetch will work on the files but we have to
> wait until all the files are read before the next processor runs. How to
> acheive this.
>
> Number of files keeps changing.
>
> Thanks,
> Asmath
