[
https://issues.apache.org/jira/browse/NIFI-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009965#comment-17009965
]
Michael Hogue commented on NIFI-6851:
-------------------------------------
I've verified backward compatibility via unit tests [1], so running a new nifi
with these changes can read old provenance events. I can't write unit tests to
verify forward compatibility (e.g. old nifi reading new provenance events, or
rollback scenario), but it looks like it'll work fine too. Here's why:
* {{RecordSchema#readFrom(InputStream)}} reads arbitrary schemas and populates
a map of fieldName -> fieldValue pairs. [2]
* The provenance event deserialization implementations all pull explicit
fieldNames out of the {{RecordSchema}} map to build
{{StandardProvenanceRecord}} objects [3]
So in a rollback scenario, events written with this new
{{FlowFileAcquisitionMethod}} field will just be constructed with that field
ignored, which is okay.
[1]
https://github.com/apache/nifi/pull/3930/files#diff-973cdae5e5e4d369ce7c2f9d321781e8R141
(note new method to create prov event w/ flow file acquisition method)
[2]
[https://github.com/apache/nifi/blob/master/nifi-commons/nifi-schema-utils/src/main/java/org/apache/nifi/repository/schema/RecordSchema.java#L103]
[3]
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/schema/LookupTableEventRecord.java#L241
> Improve Provenance RECEIVE and FETCH events to allow for specifics on how
> flowfile was RECEIVE'ed/FETCH'ed
> ----------------------------------------------------------------------------------------------------------
>
> Key: NIFI-6851
> URL: https://issues.apache.org/jira/browse/NIFI-6851
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.10.0
> Reporter: Nissim Shiman
> Assignee: Michael Hogue
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently there is no way to know if a Provenance RECEIVE or FETCH event is
> the result of an active (i.e. GetFTP, FetchFTP, GetHTTP), passive (i.e.
> ListenHTTP) or query (i.e. GetMongo) event.
> There was a discussion on the dev mailing lists regarding the most graceful
> way to do this [1] [2],
>
> At this time the smoothest path forward is to keep the current method
> signatures of receive() and fetch() as is in ProvenanceReporter.java [3]
> and to add new fetch() and receive()'s with a new parameter:
> receive(...,ACTIVE|PASSIVE|QUERY)
> If not specified it would be UNSPECIFIED.
> (Thank You [~joewitt] for this solution)
>
>
> [1] [https://mail-archives.apache.org/mod_mbox/nifi-dev/201910.mbox/browser]
> , topic of PULL ProvenanceEvent
> [2] [https://mail-archives.apache.org/mod_mbox/nifi-dev/201911.mbox/browser]
> , topic of Re: PULL ProvenanceEvent
> [3]
> [https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/provenance/ProvenanceReporter.java]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
