[
https://issues.apache.org/jira/browse/BEAM-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106619#comment-17106619
]
Luke Cwik edited comment on BEAM-9945 at 5/13/20, 7:45 PM:
-----------------------------------------------------------
[~ibzib] Can you cherrypick https://github.com/apache/beam/pull/11689 or just
the Python portion of it if there is an issue generating the cherrypick?
This would close the issue.
was (Author: lcwik):
[~ibzib] Can you cherrypick https://github.com/apache/beam/pull/11689 or just
the Python portion of it if there is an issue generating the cherrypick?
> Use consistent element count for progress counter.
> --------------------------------------------------
>
> Key: BEAM-9945
> URL: https://issues.apache.org/jira/browse/BEAM-9945
> Project: Beam
> Issue Type: Improvement
> Components: beam-model, sdk-go, sdk-java-harness, sdk-py-harness
> Reporter: Robert Bradshaw
> Assignee: Robert Bradshaw
> Priority: Critical
> Fix For: 2.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This influences how the SDK communicates progress and how splitting is
> performed.
> We currently inspect the element count of the output (which has inconsistent
> definitions across SDKs, see BEAM-9934). Instead, we can move to using an
> explicit, separate metric.
> This can currently lead to incorrect progress reporting and in some cases
> even a crash in the UW depending on the SDK, and makes it more difficult (but
> not impossible) to fix element counts in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
