Hi Peter, It hasn't been finalized yet, but we can configure it to be reasonably low. Basically the higher batch size. In regards to my question, which strategy you think will provide more throughput?
Regards, Ali On Wed, Aug 23, 2017 at 6:49 PM, Peter Wicks (pwicks) <[email protected]> wrote: > Hi Ali, > > > > How many FlowFile’s are you expecting per minute? My experience with NiFi > has been that if you have 1 FlowFile that is several GB’s in size you can > frequently process it faster than 100k FlowFile’s at a fraction the size. > There is a lot of overhead in managing the life cycle of a FlowFile when > you start to have lots of them. > > > > --Peter > > > > *From:* Ali Nazemian [mailto:[email protected]] > *Sent:* Wednesday, August 23, 2017 2:49 PM > *To:* [email protected] > *Subject:* [EXT] High-performance Nifi decoder > > > > Hi all, > > > > > > I am investigating the right approach for implementing an high-performance > network packet decoder in Nifi. I am trying to achieve about 1 Gbps > throughput on a specific decoder. (with 4 Nifi instances) I was wondering > which option would perform better. 1) Implementing my decoder in a separate > Nifi processor using NAR mechanism; 2) Implementing my decoder in C++ and > use "ExecuteStreamCommand" to run that decoder. Basically, if > ExecuteStreamCommand involves Disk for the sake of sending messages to the > written script, it doesn't matter how fast the decoder is, disk would be > the main bottleneck. > > > > > > Regards, > > Ali > -- A.Nazemian
