Hello. It's me again regarding my mergerecord question.
I still don't manage to have what I want, I may have understand how bin based processor works, it's for clarification and a question regarding performance. I want to merge a huge number of 300 octets flowfiles in 128 MB parquet file. My understanding is, for mergerecord to be able to create a bin with 128MB of data, these data must be in queue. We can't feed the bin "one flow at a time", so working with small flowfiles, I have to set the backpressure parameter to something really high, or remove completely the number of flowfile backpressure limit. I understood by reading https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache.nifi.processors.standard.MergeRecord/additionalDetails.html that it's not the "good" way to do, but I should cascade multiple merge to "slowly" make the flowfile bigger? I've made some test with a single level but I hit the "provenance recording rate". Will multiple level help? Thanks for any help. Aurélien. This electronic transmission (and any attachments thereto) is intended solely for the use of the addressee(s). It may contain confidential or legally privileged information. If you are not the intended recipient of this message, you must delete it immediately and notify the sender. Any unauthorized use or disclosure of this message is strictly prohibited. Faurecia does not guarantee the integrity of this transmission and shall therefore never be liable if the message is altered or falsified nor for any virus, interception or damage to your system.