Thanks Andy/Joe/Juan for the helpful replies. I have some follow-up
questions...
(1)
I'm not quite sure how a flowfile-level audit trail can be used for
governance. Is it expected that some system will process the info as
follows?
* for each provenance event
* if (is_bad_record(event)) then
Simon,
The provenance capability is definitely used by many users for governance and
regulatory purposes. For example, when dealing with geolocation data, many
countries regulate the export of this data outside their borders. With
provenance, you can provably demonstrate that every flowfile
Additionally it is important to note that flow level changes are now
exposed and available to reporting tasks as well. It is envisioned
this will be used to report to systems like Apache Atlas for that flow
level metadata you describe but made far more powerful by combining it
with event level
Simon,
We use NIFI's data provenance capabilities, to track the like cycle of a
"flowFile" / data object as it goes through its system lifecycle. ( LINEAGE
)
We also use it for troubleshooting as we can see the nifi attributes (
metadata ) and its content ( if configured )
You can also use
Hi All,
Can someone explain to me the business-level use cases that "provenance
events" are intended to solve?
I can see that they are useful for "flow developers" to debug problems.
But is that their only use?
Can they be used to address some kinds of regulatory compliance
requirements?
