[
https://issues.apache.org/jira/browse/APEXMALHAR-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303023#comment-15303023
]
ASF GitHub Bot commented on APEXMALHAR-2096:
--------------------------------------------
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/287#discussion_r64828460
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileSplitter.java ---
@@ -53,6 +53,7 @@
* This is a threshold on the no. of blocks emitted per window. A lot of
blocks emitted
* per window can overwhelm the downstream operators. This setting helps
to control that.
*/
+ @NotNull
--- End diff --
Is this required?
> Add blockThreshold parameter to FSInputModule
> ---------------------------------------------
>
> Key: APEXMALHAR-2096
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2096
> Project: Apache Apex Malhar
> Issue Type: Improvement
> Reporter: Priyanka Gugale
> Assignee: Priyanka Gugale
>
> FileSplitter is very fast, the downstream operators can't work at that speed,
> so we need to limit the speed with which fileSplitter works.
> One way to do is limit number of blocks emitted per window.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
