Re: [EXT] Re: Duplicate flow files *without* their content

2019-08-01 Thread Lars Winderling
> 2019 11:02 AMTo: dev@nifi.apache.org > Subject: [EXT] Re: Duplicate flow files *without* their content > Hi Edward, > thank you for your input. I didn't know about the cow-semantics, that's > really useful. I'll check out the in-depth > guide for sure!In my case, the content of

RE: [EXT] Re: Duplicate flow files *without* their content

2019-07-31 Thread Peter Wicks (pwicks)
be an empty string. Thanks, Peter From: Lars Winderling Sent: Wednesday, July 31, 2019 11:02 AM To: dev@nifi.apache.org Subject: [EXT] Re: Duplicate flow files *without* their content Hi Edward, thank you for your input. I didn't know about the cow-semantics, that's really useful. I'll check

Re: Duplicate flow files *without* their content

2019-07-31 Thread Lars Winderling
Hi Edward, thank you for your input. I didn't know about the cow-semantics, that's really useful. I'll check out the in-depth guide for sure!In my case, the content of the flow file does change heavily from one processor to the next one, so I doubt copy-on-write would help here. Best,Lars On

Re: Duplicate flow files *without* their content

2019-07-31 Thread Edward Armes
HI Lars, In short. depending on the how a FlowFile is duplicated, the content shouldn't be duplicated as well. In general, content is only duplicated when it has been deemed to have been changed (copy-on-write semantics). For the most part (unless a FlowFIle has a large number of attributes) a

Duplicate flow files *without* their content

2019-07-31 Thread Lars Winderling
Dear NiFi community, I often face the use-case where I import flow files with content of order O(1gb) or O(10gb) – already compressed. Let's day I need to branch off of a flow where the actual flow file should be processed further, and one some side branch I want just to do some kind of logging