[
https://issues.apache.org/jira/browse/BEAM-12856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Whittle updated BEAM-12856:
-------------------------------
Description:
Currently in WorkerCustomSources.java it is hard-coded to 10 seconds, 10k
elements, with 1sec waiting for elements if none are available.
There are cases where it would be beneficial to wait longer and process more
data so it would be nice if this was controlled by pipeline option.
> Allow for configuration of unbounded reader max elements, read time etc in
> StreamingDataflowRunner
> --------------------------------------------------------------------------------------------------
>
> Key: BEAM-12856
> URL: https://issues.apache.org/jira/browse/BEAM-12856
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Sam Whittle
> Assignee: Sam Whittle
> Priority: P2
>
> Currently in WorkerCustomSources.java it is hard-coded to 10 seconds, 10k
> elements, with 1sec waiting for elements if none are available.
> There are cases where it would be beneficial to wait longer and process more
> data so it would be nice if this was controlled by pipeline option.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
