[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1439: MINIFICPP-1967 Add batch processing of lines in TailFile
lordgamez commented on code in PR #1439: URL: https://github.com/apache/nifi-minifi-cpp/pull/1439#discussion_r1011717083 ## PROCESSORS.md: ## @@ -2424,6 +2424,7 @@ In the list below, the names of required properties appear in bold. Any other pr | State File | TailFileState | | Specifies the file that should be used for storing state about what data has been ingested so that upon restart NiFi can resume from where it left off | | tail-base-directory| | | Base directory used to look for files to tail. This property is required when using Multiple file mode. Can contain expression language placeholders if Attribute Provider Service is set.**Supports Expression Language: true** | | **tail-mode** | Single file | Single fileMultiple file | Specifies the tail file mode. In 'Single file' mode only a single file will be watched. In 'Multiple file' mode a regex may be used. Note that in multiple file mode we will still continue to watch for rollover on the initial set of watched files. The Regex used to locate multiple files will be run during the schedule phrase. Note that if rotated files are matched by the regex, those files will be tailed. | +| **Batch Size** | 0 | | Maximum number of lines to process in a single trigger. If set to 0 all new lines will be processed. | Review Comment: After reading the comments I don't think we can really talk about "lines" specifically in this processor, as the flow file contents are fully controlled by the delimiter property. So I'm not sure it would be a good idea to have a separate property that controls lines specifically. The `Batch Size` property name is consistently used in all processors to mark the number of flow files emitted at once by the processor so it should represent the same here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1439: MINIFICPP-1967 Add batch processing of lines in TailFile
lordgamez commented on code in PR #1439:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1439#discussion_r1011482503
##
extensions/standard-processors/processors/TailFile.cpp:
##
@@ -135,6 +135,13 @@ const core::Property TailFile::AttributeProviderService(
->asType()
->build());
+const core::Property TailFile::BatchSize(
+core::PropertyBuilder::createProperty("Batch Size")
+->withDescription("Maximum number of lines to process in a single
trigger. If set to 0 all new lines will be processed.")
Review Comment:
Good point, updated in 24f9ab5c32083fea259b4b1e0889bd93f20e8cda
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1439: MINIFICPP-1967 Add batch processing of lines in TailFile
lordgamez commented on code in PR #1439: URL: https://github.com/apache/nifi-minifi-cpp/pull/1439#discussion_r1011357368 ## PROCESSORS.md: ## @@ -2424,6 +2424,7 @@ In the list below, the names of required properties appear in bold. Any other pr | State File | TailFileState | | Specifies the file that should be used for storing state about what data has been ingested so that upon restart NiFi can resume from where it left off | | tail-base-directory| | | Base directory used to look for files to tail. This property is required when using Multiple file mode. Can contain expression language placeholders if Attribute Provider Service is set.**Supports Expression Language: true** | | **tail-mode** | Single file | Single fileMultiple file | Specifies the tail file mode. In 'Single file' mode only a single file will be watched. In 'Multiple file' mode a regex may be used. Note that in multiple file mode we will still continue to watch for rollover on the initial set of watched files. The Regex used to locate multiple files will be run during the schedule phrase. Note that if rotated files are matched by the regex, those files will be tailed. | +| **Batch Size** | 0 | | Maximum number of lines to process in a single trigger. If set to 0 all new lines will be processed. | Review Comment: If the delimiter already control how the lines are processed, is it okay if we only change the description as @adamdebreceni suggested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
