lostluck commented on code in PR #26374:
URL: https://github.com/apache/beam/pull/26374#discussion_r1175435220
##########
sdks/go/pkg/beam/core/runtime/exec/plan.go:
##########
@@ -38,7 +39,7 @@ type Plan struct {
bf *bundleFinalizer
checkpoints []*Checkpoint
- status Status
+ status Status // Uses atomic getter and setter to avoid dataraces on
Splits.
Review Comment:
Splits are how Beam (and thus Dataflow) allow for Liquid Sharding. That is,
if one bundle is taking a long time, the runner can request the SDK to give up
some fraction of it's remaining work so it can be processed in parallel. (This
would either be via channel or sub-element splitting with SDFs, but that's a
different topic.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
