Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2448#discussion_r171280456 --- Diff: nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java --- @@ -129,26 +144,44 @@ .addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR) .build(); static final PropertyDescriptor RESULTS_PER_FLOWFILE = new PropertyDescriptor.Builder() - .name("results-per-flowfile") - .displayName("Results Per FlowFile") - .description("How many results to put into a flowfile at once. The whole body will be treated as a JSON array of results.") - .required(false) - .expressionLanguageSupported(true) - .addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR) - .build(); + .name("results-per-flowfile") + .displayName("Results Per FlowFile") + .description("How many results to put into a flowfile at once. The whole body will be treated as a JSON array of results.") + .required(false) + .expressionLanguageSupported(true) + .addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR) + .build(); + static final PropertyDescriptor ESTIMATE_PROGRESS = new PropertyDescriptor.Builder() + .name("estimate-progress") + .displayName("Estimate Progress") + .description("If enabled, a count query will be run first, using the configured query, and attributes will be added to each flowfile showing how far they are into the result set.") + .required(true) + .addValidator(StandardValidators.BOOLEAN_VALIDATOR) + .allowableValues(GM_TRUE, GM_FALSE) + .defaultValue(GM_FALSE.getValue()) + .build(); + static final PropertyDescriptor PROGRESSIVE_COMMITS = new PropertyDescriptor.Builder() + .name("progressive-commits") + .displayName("Commit After Each Batch") --- End diff -- I'm a little confused here about the term "batch". It doesn't seem directly related to the Batch Size property (since the latter is kind of a server-side thing, like a JDBC "fetch size"?), and in the code a "batch" seems to refer to the number of files set in Results Per Flowfile. Can you explain a little more about what's going on with the progressive commits? If I have Results per Flowfile set to 100 and Batch Size set to 1000, would I get 10 flow files committed at once as once "batch"? Or is it always one commit per flowfile (if Commit After Each Batch is set)?
---