gemini-code-assist[bot] commented on code in PR #38523:
URL: https://github.com/apache/beam/pull/38523#discussion_r3276336584
##########
sdks/go/pkg/beam/runners/prism/internal/stage.go:
##########
@@ -240,7 +240,7 @@ progress:
sr, err := b.Split(ctx, wk, 0.5 /* fraction of
remainder */, nil /* allowed splits */)
if err != nil {
slog.Warn("SDK Error from split,
aborting splits and failing bundle", "bundle", rb, "error", err.Error())
- if b.BundleErr != nil {
+ if b.BundleErr == nil {
b.BundleErr = err
}
Review Comment:
Accessing and setting `b.BundleErr` here is a data race, as this field is
also modified in the worker's response handler (`Respond`) in a different
goroutine. Since the `B` struct does not appear to have internal
synchronization for this field, you should use a mutex or other synchronization
primitive to ensure thread-safe access.
##########
sdks/go/pkg/beam/runners/prism/internal/worker/bundle.go:
##########
@@ -102,7 +104,11 @@ func (b *B) Respond(resp *fnpb.InstructionResponse) {
return
}
b.responded = true
+ if b.Done != nil {
+ close(b.Done)
+ }
if resp.GetError() != "" {
+ slog.Error("DEBUG: Prism received bundle error from worker
response", "bundle", resp.GetInstructionId())
b.BundleErr = fmt.Errorf("bundle %v %v failed:%v",
resp.GetInstructionId(), b.PBDID, resp.GetError())
Review Comment:
There is a race condition here: `b.Done` is closed before `b.BundleErr` is
set. A consumer waiting on `b.Done` (e.g., in `ProcessOn`) might see the
channel closure and proceed to check `b.BundleErr` before it has been assigned
the actual error from the worker. This could lead to a failed bundle being
incorrectly treated as successful. Additionally, the log message contains a
"DEBUG:" prefix while being at the `Error` level, which is confusing and should
be cleaned up.
```suggestion
if resp.GetError() != "" {
slog.Error("Prism received bundle error from worker response",
"bundle", resp.GetInstructionId())
b.BundleErr = fmt.Errorf("bundle %v %v failed:%v",
resp.GetInstructionId(), b.PBDID, resp.GetError())
}
if b.Done != nil {
close(b.Done)
}
if resp.GetError() != "" {
```
##########
sdks/go/pkg/beam/runners/prism/internal/testdofns_test.go:
##########
@@ -404,3 +405,37 @@ func (fn *selfCheckpointingDoFn) ProcessElement(rt
*sdf.LockRTracker, _ []byte,
}
}
}
+
+type errorSplitTracker struct {
+ *offsetrange.Tracker
+}
+
+func (t *errorSplitTracker) TrySplit(fraction float64) (any, any, error) {
+ return nil, nil, fmt.Errorf("intentional split error from tracker")
+}
+
+type slowFailSDF struct{}
+
+func (fn *slowFailSDF) CreateInitialRestriction(config SourceConfig)
offsetrange.Restriction {
+ return offsetrange.Restriction{Start: 0, End: config.NumElements}
+}
+
+func (fn *slowFailSDF) SplitRestriction(config SourceConfig, rest
offsetrange.Restriction) []offsetrange.Restriction {
+ return rest.EvenSplits(config.InitialSplits)
+}
+
+func (fn *slowFailSDF) RestrictionSize(_ SourceConfig, rest
offsetrange.Restriction) float64 {
+ return rest.Size()
+}
+
+func (fn *slowFailSDF) CreateTracker(rest offsetrange.Restriction)
*sdf.LockRTracker {
+ return sdf.NewLockRTracker(&errorSplitTracker{Tracker:
offsetrange.NewTracker(rest)})
+}
+
+func (fn *slowFailSDF) ProcessElement(ctx context.Context, rt
*sdf.LockRTracker, config SourceConfig, emit func(int64)) error {
+ fmt.Println("DEBUG: slowFailSDF.ProcessElement invoked")
Review Comment:
This `fmt.Println` appears to be a leftover debug statement and should be
removed to keep the test output clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
