[
https://issues.apache.org/jira/browse/BEAM-14484?focusedWorklogId=773739&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773739
]
ASF GitHub Bot logged work on BEAM-14484:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/May/22 21:11
Start Date: 23/May/22 21:11
Worklog Time Spent: 10m
Work Description: lostluck commented on code in PR #17724:
URL: https://github.com/apache/beam/pull/17724#discussion_r879869762
##########
sdks/go/pkg/beam/core/sdf/sdf.go:
##########
@@ -72,9 +72,12 @@ type RTracker interface {
// the only split point is the end of the restriction, or the split
failed for some recoverable
// reason), then this function returns nil as the residual.
//
- // If the split fraction is 0 (e.g. a self-checkpointing split)
TrySplit() should return either
- // a nil primary or an RTracker that is both bounded and has size 0.
This ensures that there is
- // no data that is lost by not being rescheduled for execution later.
+ // If the split fraction is 0 (e.g. a self-checkpointing split)
TrySplit() should return
+ // a primary restriction that represents no remaining work, and the
residual should
+ // contain all remaining work. The RTracker should be marked as done
+ // (and return true when IsDone() is called) after that split.
+ // This will ensure that there is no data loss, which would result in
+ // the pipeline failing during the checkpoint.
Review Comment:
```suggestion
// If the split fraction is 0 (e.g. a self-checkpointing split)
TrySplit() should return
// a primary restriction that represents no remaining work, and the
residual should
// contain all remaining work. The RTracker should be marked as done
// (and return true when IsDone() is called) after that split.
// This will ensure that there is no data loss, which would result in
// the pipeline failing during the checkpoint.
```
Issue Time Tracking
-------------------
Worklog Id: (was: 773739)
Time Spent: 8h (was: 7h 50m)
> Improve error message surrounding primary returns in the self-checkpointing
> code
> --------------------------------------------------------------------------------
>
> Key: BEAM-14484
> URL: https://issues.apache.org/jira/browse/BEAM-14484
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Jack McCluskey
> Assignee: Jack McCluskey
> Priority: P1
> Time Spent: 8h
> Remaining Estimate: 0h
>
> The error message in the Go SDK harness around returned primaries in the
> self-checkpointing code
> ([https://github.com/apache/beam/blob/ea1f292e9cf31fc8c4803b10d811f0d3ee184ae7/sdks/go/pkg/beam/core/runtime/exec/datasource.go#L375)]
> is unclear and should be made more explicit. It should also guide the user
> towards making sure that the restriction behaves properly in the
> self-checkpointing case.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
