[
https://issues.apache.org/jira/browse/BEAM-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105926#comment-17105926
]
Luke Cwik edited comment on BEAM-9945 at 5/13/20, 3:23 AM:
---
When testing the implementation against runner v2, I found that the Go SDK's
read index ends at *# elements* or the *stop index* when splitting while the
Python/Java implementations always stop at *# elements - 1* or *stop index - 1*
I think it makes sense to use Go's definition.
was (Author: lcwik):
When testing the implementation against runner v2, I found that the Go SDK's
read index ends at *# elements* or the *stop index* when splitting while the
Python/Java implements always stop at *# elements - 1* or *stop index - 1*
I think it makes sense to use Go's definition.
> Use consistent element count for progress counter.
> --
>
> Key: BEAM-9945
> URL: https://issues.apache.org/jira/browse/BEAM-9945
> Project: Beam
> Issue Type: Improvement
> Components: beam-model, sdk-go, sdk-java-harness, sdk-py-harness
>Reporter: Robert Bradshaw
>Assignee: Robert Bradshaw
>Priority: Critical
> Fix For: 2.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This influences how the SDK communicates progress and how splitting is
> performed.
> We currently inspect the element count of the output (which has inconsistent
> definitions across SDKs, see BEAM-9934). Instead, we can move to using an
> explicit, separate metric.
> This can currently lead to incorrect progress reporting and in some cases
> even a crash in the UW depending on the SDK, and makes it more difficult (but
> not impossible) to fix element counts in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
