[
https://issues.apache.org/jira/browse/BEAM-5791?focusedWorklogId=248353&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-248353
]
ASF GitHub Bot logged work on BEAM-5791:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/May/19 23:07
Start Date: 24/May/19 23:07
Worklog Time Spent: 10m
Work Description: ananvay commented on issue #8683: [BEAM-5791] splitting
support over FnAPI for Go SDK
URL: https://github.com/apache/beam/pull/8683#issuecomment-495810496
https://issues.apache.org/jira/browse/BEAM-5791
This change is the "unreading elements already sent by the runner" part.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 248353)
Time Spent: 3.5h (was: 3h 20m)
> Bound the amount of data on the data plane by time.
> ---------------------------------------------------
>
> Key: BEAM-5791
> URL: https://issues.apache.org/jira/browse/BEAM-5791
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow, sdk-java-harness, sdk-py-harness
> Reporter: Robert Bradshaw
> Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> This is especially important for Fn API reads, where each element represents
> a shard to read and may be very expensive, but many elements may be waiting
> in the Fn API buffer.
> The need for this will be mitigated with full SDF support for liquid sharding
> over the Fn API, but not eliminated unless the runner can "unread" elements
> it has already sent.
> This is especially important in for dataflow jobs that start out small but
> then detect that they need more workers (e.g. due to the initial inputs being
> an SDF).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
