persicoana opened a new pull request, #40820: URL: https://github.com/apache/beam/pull/40820
Fixes #40818. Prism currently admits expired aggregation input when pending work in a later window holds the stage output watermark behind upstream progress. Compare window completion to the existing minimum upstream PCollection watermark instead. Keep the stage's internal/output holds and the separate stateful admission path unchanged. Add five engine regression controls: expired input with held output, within allowed lateness, exact completion, past completion, and the minimum of two upstream inputs. Preserve the later window's pending data and both internal holds in every case. Add a 2.78.0 unreleased bugfix entry. Validation against master `49a6a2ecfa4679c025e434deb90293dc9cfee042`, Go 1.26.2, Linux x86_64: - The regression alone fails in the two expired-input child cases before the production change; the other three boundary controls pass. - `go test -mod=readonly -p=2 -count=1 -timeout=5m ./go/pkg/beam/runners/prism/internal/engine ./go/pkg/beam/runners/prism/internal/jobservices` passes. - `go test -mod=readonly -p=2 -count=1 -race -timeout=5m ./go/pkg/beam/runners/prism/internal/engine` passes with no race report. - The self-contained Python 2.77.0 SDK TestStream reproducer in #40818 was executed against independently built original/patched master Prism binaries. Both jobs reach DONE. Original physically emits expired A=100 twice plus B=4's on-time/closing panes; patched emits only B's two panes and passes the semantic oracle. - New Go test is gofmt-clean; upstream go.mod/go.sum remain unchanged. No new dependencies. These are selected local checks, not a claim that all Beam suites/CI or other stateful, cancellation, sink or runner profiles are qualified. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
