For manual checks, NiFi provides the provenance data. You can check the last processor in the flow to see recent flowfiles and their lineage. For a more systematic approach you can use any number of external systems to help you track this, depending on what’s available.
The simplest is to use LogMessage processors and optionally configure Logback to send those to a different logfile. You’re most likely already using a log analyzer stack, so a query there should give you the insight you need. Another way I have good experience with is to use a database table. Insert a record when the process starts (using PutSQL) and then update it with either success or failure and timestamps. That has the added benefit of being able to retrieve the last run’s status from the NiFi flow. Regards, Isha Van: K Makeev <[email protected]> Verzonden: woensdag 15 juni 2022 19:05 Aan: [email protected] Onderwerp: batch job state tracking Hi Let's say I have a flow which reads the whole database table and puts all its content to another database table. It runs in a batch mode - once a day. How can I track its state - like when it was completed? I know that Nifi is a stream thing but may there some new features to deal with it Thank you Kirill
