[
https://issues.apache.org/jira/browse/NIFI-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352494#comment-16352494
]
ASF GitHub Bot commented on NIFI-4838:
--------------------------------------
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2448#discussion_r165991021
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java
---
@@ -51,14 +53,22 @@
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Collections;
+import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
+import java.util.Map;
import java.util.Set;
@Tags({ "mongodb", "read", "get" })
@InputRequirement(Requirement.INPUT_FORBIDDEN)
@CapabilityDescription("Creates FlowFiles from documents in MongoDB")
+@WritesAttributes( value = {
+ @WritesAttribute(attribute = "progress.estimate", description = "The
estimated total documents that match the query. Written if estimation is
enabled."),
+ @WritesAttribute(attribute = "progress.segment.start", description =
"Where the first part of the segment is in the total result set. Written if
estimation is enabled."),
+ @WritesAttribute(attribute = "progress.segment.end", description =
"Where the last part of the segment is in the total result set. Written if
estimation is enabled."),
+ @WritesAttribute(attribute = "progress.index", description = "When
results are written one-by-one to flowfiles, this is is set to indicate
estimated progress. Written if estimation is enabled.")
--- End diff --
Typo: this is <del>is</del> set
> Make GetMongo support multiple commits and give some progress indication
> ------------------------------------------------------------------------
>
> Key: NIFI-4838
> URL: https://issues.apache.org/jira/browse/NIFI-4838
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Mike Thomsen
> Assignee: Mike Thomsen
> Priority: Major
>
> It shouldn't wait until the end to do a commit() call because the effect is
> that GetMongo looks like it has hung to a user who is pulling a very large
> data set.
> It should also have an option for running a count query to get the current
> approximate count of documents that would match the query and append an
> attribute that indicates where a flowfile stands in the total result count.
> Ex:
> query.progress.point.start = 2500
> query.progress.point.end = 5000
> query.count.estimate = 17,568,231
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
