Sebastian Rothbucher created NIFI-11127:
-------------------------------------------
Summary: PutMongoBulk processor - use the more efficient bulkWrite
API
Key: NIFI-11127
URL: https://issues.apache.org/jira/browse/NIFI-11127
Project: Apache NiFi
Issue Type: New Feature
Reporter: Sebastian Rothbucher
Problem to solve: PutMongo can only update (or upsert) one record at a time.
For thousands of records, this is not efficient. The alternative PutMongoRecord
assumes a lot about the documents written
Proposed solution: add a PutMongoBulk processor that uses the bulkWrite API -
bundle operations together into one (can be hundreds at a time, also by merging
flowfiles). Can be any insert / replace / update / upsert with no assumption
about how the documents look like. MongoDB processes all in one operation -
resulting in _way_ less load and way more efficient operation.
What I can bring: I wrote such a processor about a year back (for earlier
version); it did work in production flawlessly for months now. I did update it
to 1.20-SNAPSHOT and re-tested; can submit a PR immediately after I get a
ticket number here. Tried to be similar to PutMongo wherever poss; feedback is
sure appreciated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
