github-actions[bot] commented on code in PR #41443:
URL: https://github.com/apache/beam/pull/41443#discussion_r4607086059
##########
sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go:
##########
@@ -1486,6 +1492,9 @@ func (ss *stageState) injectTriggeredBundlesIfReady(em
*ElementManager, window t
if ready {
state.Pane = computeNextTriggeredPane(state.Pane,
endOfWindowReached)
+ if ss.strat.Trigger != nil &&
state.getTriggerState(ss.strat.Trigger).finished {
Review Comment:
**🔴 Critical**: When the root trigger finishes in `Accumulating` mode,
setting `state.Pane.IsLast = true` here (and in `buildProcessingTimeBundle` at
line 2336) marks the triggered pane as final, but `buildTriggeredBundle` (line
1756) still checks only `if ss.strat.Accumulating` and pushes `toProcess` back
onto `dnt.elements`. When the watermark later advances past the end of the
window, `buildEventTimeBundle` pops those re-queued elements and emits an extra
pane after a pane was already emitted with `IsLast = true`.
`buildTriggeredBundle` should avoid re-queuing elements onto `dnt.elements`
when the root trigger has finished / `state.Pane.IsLast` is true (mirroring the
`if ss.strat.Accumulating && !state.Pane.IsLast` check in
`buildEventTimeBundle`), and `buildProcessingTimeBundle` should persist
`ss.state[LinkID{}][e.window][key] = state` before calling
`ss.buildTriggeredBundle` so it sees the updated state.
##########
sdks/go/pkg/beam/runners/prism/internal/engine/strategy.go:
##########
@@ -466,6 +527,23 @@ func (t *TriggerOrFinally) onFire(state *StateData) {
}
}
+func (t *TriggerOrFinally) onWatermarkFire(state *StateData) {
+ if state.getTriggerState(t).finished {
+ return
+ }
+ if eow, ok := t.Finally.(*TriggerAfterEndOfWindow); ok {
Review Comment:
**🟡 Medium**: `TriggerOrFinally.onWatermarkFire` only invokes
`onWatermarkFire` on `t.Finally` when it is directly a
`*TriggerAfterEndOfWindow`. If `t.Finally` is a composite trigger (such as
`TriggerAfterAll`, `TriggerAfterAny`, or `TriggerAfterEach` containing an
`AfterEndOfWindow`) or a leaf trigger with uncommitted state
(`TriggerElementCount`, `TriggerAfterProcessingTime`),
`t.Finally.onWatermarkFire(state)` is skipped. Calling
`t.Finally.onWatermarkFire(state)` through the `Trigger` interface (matching
`onElement`, `shouldFire`, `onFire`, and `reset`) handles
`*TriggerAfterEndOfWindow` identically while supporting all other trigger types:
```go
if t.Finally != nil {
t.Finally.onWatermarkFire(state)
if state.getTriggerState(t.Finally).finished {
triggerClearAndFinish(t, state)
return
}
}
```
##########
sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go:
##########
@@ -2321,14 +2333,21 @@ func (*aggregateStageKind) buildProcessingTimeBundle(ss
*stageState, em *Element
if ready {
state.Pane = computeNextTriggeredPane(state.Pane,
endOfWindowReached)
+ if ss.strat.Trigger != nil &&
state.getTriggerState(ss.strat.Trigger).finished {
+ state.Pane.IsLast = true
+ }
// We're going to process this trigger!
- elems, pendingAdjustment = ss.buildTriggeredBundle(em,
string(e.keyBytes), e.window)
+ elems, pendingAdjustment = ss.buildTriggeredBundle(em,
key, e.window)
Review Comment:
**🟠High**: When a processing-time timer fires, the pending elements for
`(key, e.window)` may have already been emitted by an earlier watermark or
trigger bundle, causing `ss.buildTriggeredBundle` to return `len(elems) == 0`.
Mutating `state` via `IsTriggerReady` and `computeNextTriggeredPane`,
persisting `ss.state[LinkID{}][e.window][key] = state`, and appending to
`panesInBundle` when no elements are pending for `e.window` advances
`state.Pane.Index` (and can mark the trigger finished) without emitting a pane,
shifting pane indices on subsequent late firings.
Check that `ss.pendingByKeys[key]` still has pending elements for `e.window`
before advancing trigger and pane state (or only commit `state.Pane` and append
to `panesInBundle` when `len(elems) > 0`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]