Re: Provenance events without FlowFiles?

2023-11-03 Thread Lehel Boér
e only one in the StandardProvenanceEventRecord. 3. Are the ProvenanceEventRecords expandable with additional fields? 1. From: Matt Burgess Sent: Friday, October 27, 2023 0:20 To: dev@nifi.apache.org Subject: Re: Provenance events without FlowFiles? I won't speak for Mark

Re: Provenance events without FlowFiles?

2023-10-26 Thread Matt Burgess
I won't speak for Mark but I took that to mean that you pass in a generic Java (or custom) File object (no matter what or where it points to) so the ProvenanceReporter can use the appropriate information from that object (like size) to populate the provenance event, not any real reference to the

Re: Provenance events without FlowFiles?

2023-10-26 Thread Pierre Villard
> > Rather, I’d say it's an UPLOAD_FILE event. So I’d lean more toward an > uploadFile() method on ProvenanceReporter that takes as an argument a > `File` (as well as a FlowFile). The size would come from the File itself, > and the event would convey the information about the local file that was >

Re: Provenance events without FlowFiles?

2023-10-26 Thread Matt Burgess
AFAIK it is fine and appropriate to issue multiple provenance events for a single FlowFile. In the case for PutAzureBlobStorage uploading a file to Azure, it is the incoming FlowFile that triggers the upload. Before reporting a provenance event, attributes are added to the FlowFile, so that

Re: Provenance events without FlowFiles?

2023-10-26 Thread Mark Payne
Lehel, I don’t believe we should be trying to create a “Mock FlowFile.” I am ok with an update to the ProvenanceReporter interface. But I don’t think it should accept a “size” parameter. Rather, I think this is a completely different type of event that is occurring. This is not a “send” in

Provenance events without FlowFiles?

2023-10-26 Thread Lehel Boér
Hi everyone, I would like to address a particular scenario that has recently come to my attention regarding the use of the PutAzureBlobStorage processor with the FileResourceService. When the PutAzureBlobStorage processor is used with the FileResourceService, it currently uploads a file from