Additionally it is important to note that flow level changes are now exposed and available to reporting tasks as well. It is envisioned this will be used to report to systems like Apache Atlas for that flow level metadata you describe but made far more powerful by combining it with event level lineage as well.
On Wed, Apr 19, 2017 at 5:23 PM, Juan Sequeiros <[email protected]> wrote: > Simon, > > We use NIFI's data provenance capabilities, to track the like cycle of a > "flowFile" / data object as it goes through its system lifecycle. ( LINEAGE > ) > We also use it for troubleshooting as we can see the nifi attributes ( > metadata ) and its content ( if configured ) > > You can also use provenance to "replay" your data at specific points during > its dataflow life cycle. > > Please reference similar answer given on stackoverflow by Joe Witt [1] > I also recommend reading Apache NIFI in depth which has a good provenance > section [2] > > [1] > http://stackoverflow.com/questions/38948494/what-is-the-purpose-of-data-provenance-in-apache-nifi-processors > [2] https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html > > > > On Wed, Apr 19, 2017 at 6:02 AM <[email protected]> wrote: >> >> Hi All, >> >> Can someone explain to me the business-level use cases that "provenance >> events" are intended to solve? >> >> I can see that they are useful for "flow developers" to debug problems. >> But is that their only use? >> >> Can they be used to address some kinds of regulatory compliance >> requirements? Or data governance issues? Such problems however generally >> need information at the _flow_ level, not at the per-message level.. >> >> Thanks in advance, >> Simon
