[
https://issues.apache.org/jira/browse/BEAM-11910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yichi Zhang updated BEAM-11910:
-------------------------------
Status: Open (was: Triage Needed)
> Increase subsequent page size for bags after the first
> ------------------------------------------------------
>
> Key: BEAM-11910
> URL: https://issues.apache.org/jira/browse/BEAM-11910
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Sam Whittle
> Assignee: Sam Whittle
> Priority: P2
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently the page size of bags requested from the streaming dataflow backend
> is always 8MB. In pipelines with large bags this can limit throughput as it
> results in more round-trips to the backend. In particular with Streaming
> Engine this is noticable due to increased latency.
> I propose using 8MB for the first bag fetch and then doubling the limit for
> subsequent paginations
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
