Thanks Mark. That makes sense. Regards, Ben
From: Mark Payne Reply-To: "[email protected]<mailto:[email protected]>" Date: Thursday, October 15, 2015 at 1:09 PM To: "[email protected]<mailto:[email protected]>" Subject: Re: Provenance doesn't work with FetchS3Object Ben, Since FetchS3Object is not creating the FlowFile, it should not be emitting a RECEIVE event. This is certainly a bug. I have created a ticket for this: https://issues.apache.org/jira/browse/NIFI-1038 Thanks -Mark On Oct 15, 2015, at 3:57 PM, Ben Meng <[email protected]<mailto:[email protected]>> wrote: Oleg, Yes, I’ve tried running FetchS3Object without any incoming FlowFile, and it just didn’t generate any output. I’ve also confirmed the behavior by inspecting the codes. The first thing it does is to check if there’s an existing FlowFile, and returns if there isn’t. Regards, Ben From: Oleg Zhurakousky Reply-To: "[email protected]<mailto:[email protected]>" Date: Thursday, October 15, 2015 at 12:49 PM To: "[email protected]<mailto:[email protected]>" Subject: Re: Provenance doesn't work with FetchS3Object Ben I don’t think it needs an incoming FlowFile. It is a scheduled component and will retrieve contents based on how you configure scheduling. Have you tried it without incoming FlowFiles? Cheers Oleg On Oct 15, 2015, at 3:38 PM, Ben Meng <[email protected]<mailto:[email protected]>> wrote: I understand that FetchS3Object processor requires an incoming FlowFile to trigger it. The problem is that FetchS3Object emits a RECEIVE provenance event for the existing FlowFile. That event causes following error when I try to open the lineage chart for a simple flow: GenerateFlowFile -> FetchS3Object. "Found cycle in graph. This indicates that multiple events were registered claiming to have generated the same FlowFile (UUID = 40f58407-ea10-4843-b8d1-be0e24f685aa)" Should FetchS3Object create a new FlowFile for each fetched object? If so, does it really require an incoming FlowFile? Regards, Ben ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________
