Mike Thomsen created NIFI-4122:
----------------------------------

             Summary: GetMongo should be able to group results into a set of 
flowfiles
                 Key: NIFI-4122
                 URL: https://issues.apache.org/jira/browse/NIFI-4122
             Project: Apache NiFi
          Issue Type: Improvement
            Reporter: Mike Thomsen
            Priority: Minor


GetMongo should be able to take a user-defined limit and group results by that 
size into flowfiles rather than having only the ability to do a 1:1 
relationship between result and flowfile.

For example, if the user specifies 100, 100 results should be grouped together 
and turned into a JSON array that can be broken up later as needed.

This need arose when doing a bulk data ingestion from Mongo. We had shy of 400k 
documents, and the 1:1 generation of flowfiles blew right through our limits on 
the content repository. Adding this feature would make it feasible to control 
that sort of behavior more thoroughly for events like bulk ingestion.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to