Hey guys, I've been asked to put together a flow that effectively forms a DAG with a set of time-based dependencies.
What this means is that thought the DAG _could_ complete immediately, I need to add a wait condition on the detection of each file, with either time-elapsed or time-since-epoch being the trigger. For example, I may have to wait one hour after a file is received before triggering a process, or if nothing has changed then trigger the process at 5pm. I'll also need to create batch-id for each time the process is triggered. Is this within the use-case for Nifi; or should I be looking at other projects, such as airflow? Recommendations would be welcomed. Regards, Andy Loughran