tkaymak commented on PR #39576: URL: https://github.com/apache/beam/pull/39576#issuecomment-5341477524
R: @Abacn for the runner and build mechanics, you reviewed the Spark 4 batch runner PRs this builds on. R: @kennknowles for the Beam model side, watermark propagation, trigger semantics and the state and timer bridge on transformWithState. The most interesting finding for you is probably the timer off by one described in the PR, Spark fires a wake up at expiry equal to the watermark while the Beam model requires the watermark to be strictly past the timer, which silently dropped the on time pane until we bounded the fire set. This is a draft to get agreement on the direction first, the plan is to split it into five smaller PRs afterwards. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
