[jira] [Updated] (BEAM-9945) Use consistent element count for progress counter.
[ https://issues.apache.org/jira/browse/BEAM-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-9945: Status: Open (was: Triage Needed) > Use consistent element count for progress counter. > -- > > Key: BEAM-9945 > URL: https://issues.apache.org/jira/browse/BEAM-9945 > Project: Beam > Issue Type: Improvement > Components: beam-model, sdk-go, sdk-java-harness, sdk-py-harness >Reporter: Robert Bradshaw >Assignee: Robert Bradshaw >Priority: Critical > Fix For: 2.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This influences how the SDK communicates progress and how splitting is > performed. > We currently inspect the element count of the output (which has inconsistent > definitions across SDKs, see BEAM-9934). Instead, we can move to using an > explicit, separate metric. > This can currently lead to incorrect progress reporting and in some cases > even a crash in the UW depending on the SDK, and makes it more difficult (but > not impossible) to fix element counts in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9945) Use consistent element count for progress counter.
[ https://issues.apache.org/jira/browse/BEAM-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-9945: Status: Open (was: Triage Needed) > Use consistent element count for progress counter. > -- > > Key: BEAM-9945 > URL: https://issues.apache.org/jira/browse/BEAM-9945 > Project: Beam > Issue Type: Improvement > Components: beam-model, sdk-go, sdk-java-harness, sdk-py-harness >Reporter: Robert Bradshaw >Assignee: Robert Bradshaw >Priority: Critical > Fix For: 2.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This influences how the SDK communicates progress and how splitting is > performed. > We currently inspect the element count of the output (which has inconsistent > definitions across SDKs, see BEAM-9934). Instead, we can move to using an > explicit, separate metric. > This can currently lead to incorrect progress reporting and in some cases > even a crash in the UW depending on the SDK, and makes it more difficult (but > not impossible) to fix element counts in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)
