[
https://issues.apache.org/jira/browse/NIFI-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128911#comment-15128911
]
Mark Payne commented on NIFI-1018:
----------------------------------
Hmm, I like the concept, but I think the idea needs more iteration here. The
Controller Service can indicate that it 'received' the data - but then it would
need a FlowFile to report as the 'thing received'. Then, when that dataset is
used, the Controller Service has no real way most of the time of reporting the
FlowFile that it was used against (because the Controller Service API would
generally be a lookup type of mechanism, though it doesn't have to be).
Perhaps another way to look at this is to have the ControllerService of
interest expose a Dataset URI or something of that nature, and then have the
Processor that updates the FlowFile report this, via an ENRICH Provenance event
maybe? Something of that nature, that could indicate that this FlowFIle was
'enriched' or 'joined' or whatever with this dataset.
> Allow ControllerServices access to ProvenanceReporter
> -----------------------------------------------------
>
> Key: NIFI-1018
> URL: https://issues.apache.org/jira/browse/NIFI-1018
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Brandon DeVries
> Assignee: Michael Moser
>
> Currently we maintain a provenance trail for all files flowing through NiFi
> Processors. However, if a ControllerService uses some data set it generally
> just loads it from disk after it is fetched using a normal NiFi flow.
> However, this breaks the provenance trail for the data set... there is no way
> (in provenance terms) of knowing what data set the ControllerService is using
> or when it was loaded. By giving ControllerServices access to the
> ProvenanceReporter, they can acknowledge "receipt" of a data set, so the
> provenance trail from pull to use is maintained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
