Robert Bradshaw created BEAM-5791:
-------------------------------------
Summary: Bound the amount of data on the data plane by time.
Key: BEAM-5791
URL: https://issues.apache.org/jira/browse/BEAM-5791
Project: Beam
Issue Type: Improvement
Components: runner-dataflow, sdk-java-harness, sdk-py-harness
Reporter: Robert Bradshaw
Assignee: Henning Rohde
This is especially important for Fn API reads, where each element represents a
shard to read and may be very expensive, but many elements may be waiting in
the Fn API buffer.
The need for this will be mitigated with full SDF support for liquid sharding
over the Fn API, but not eliminated unless the runner can "unread" elements it
has already sent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
