[
https://issues.apache.org/jira/browse/NIFI-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518355#comment-16518355
]
ASF GitHub Bot commented on NIFI-4838:
--------------------------------------
Github user MikeThomsen commented on the issue:
https://github.com/apache/nifi/pull/2448
With your second option, if you don't transfer the original input before
you call `session.commit()` it will throw an exception because the original
input will not be assigned to a relationship. So I am not sure if your second
option is possible if I am understanding it correctly since it seems to rely on
keeping the original flowfile around for each set of results and using
`create(FlowFile)` to set a parent relationship.
if incremental commits are used, would we want to dynamically remove the
original relationship or keep it in case the user changes their mind so they
don't have to reconfigure flows as they experiment?
> Make GetMongo support multiple commits and give some progress indication
> ------------------------------------------------------------------------
>
> Key: NIFI-4838
> URL: https://issues.apache.org/jira/browse/NIFI-4838
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Mike Thomsen
> Assignee: Mike Thomsen
> Priority: Major
>
> It shouldn't wait until the end to do a commit() call because the effect is
> that GetMongo looks like it has hung to a user who is pulling a very large
> data set.
> It should also have an option for running a count query to get the current
> approximate count of documents that would match the query and append an
> attribute that indicates where a flowfile stands in the total result count.
> Ex:
> query.progress.point.start = 2500
> query.progress.point.end = 5000
> query.count.estimate = 17,568,231
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
