[
https://issues.apache.org/jira/browse/BEAM-14469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jack McCluskey updated BEAM-14469:
----------------------------------
Description:
The TestCheckpointing integration test produces a large number of zero-length
restrictions on split, leading to extremely verbose logging and errors when run
on dataflow/with multiple runners.
This is tied to the offsetrange tracker producing zero-length restrictions on
TrySplit() calls in edge cases. The unintended usage of the tracker in this
streaming context creates this issue but did reveal that TrySplit() calls were
not allowing nil primary returns, which could cause issues moving forward.
was:The TestCheckpointing integration test produces a large number of
zero-length restrictions on split, leading to extremely verbose logging and
errors when run on dataflow/with multiple runners. Reducing the number of
zero-length restrictions produced in the test should alleviate this problem.
> TestCheckpointing produces too many zero-length restrictions
> ------------------------------------------------------------
>
> Key: BEAM-14469
> URL: https://issues.apache.org/jira/browse/BEAM-14469
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Jack McCluskey
> Assignee: Jack McCluskey
> Priority: P2
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The TestCheckpointing integration test produces a large number of zero-length
> restrictions on split, leading to extremely verbose logging and errors when
> run on dataflow/with multiple runners.
>
> This is tied to the offsetrange tracker producing zero-length restrictions on
> TrySplit() calls in edge cases. The unintended usage of the tracker in this
> streaming context creates this issue but did reveal that TrySplit() calls
> were not allowing nil primary returns, which could cause issues moving
> forward.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
