Arpad Boda created MINIFICPP-1062:
-------------------------------------
Summary: Flowfile repo performance is unacceptable
Key: MINIFICPP-1062
URL: https://issues.apache.org/jira/browse/MINIFICPP-1062
Project: Apache NiFi MiNiFi C++
Issue Type: Improvement
Affects Versions: 0.6.0
Reporter: Arpad Boda
Assignee: Arpad Boda
Fix For: 0.7.0
In case a processor produces high number of flowfiles in big batches (50+ in
one onTrigger call), MiNiFi performance is ridiculous, the max throughput is at
about 100FF/s on a decent notebook.
The bottleneck is related to the way ProcessSession and Connection handles new
flow records: every record is added to flowfile repo in a new transaction.
As rocksdb supports batching, flowfiles created in one batch should be added as
a single transaction.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
