shunping commented on issue #35771: URL: https://github.com/apache/beam/issues/35771#issuecomment-3146172729
After discussing with @lostluck, the key of this problem is bundle splitting in prism. If we set `OneKeyPerBundle` to true, the pipeline can run fine. Originally, I thought changing the code in `SplitBundle` to address the change of stage state when split happens can solve the problem. However, I then uncover some deeper problem in Python SDK. 1) When Prism sends out a split request to a SDK worker, it sends it out to the input transform of that stage, e.g. `stage-006-source` in our above example. There are three timers firing at the end of the global windows, so it assumes it can split. 2) When SDK worker receives this, it accepts the splitting. 3) However, when the worker is processing timers, it is actually not going through "stage-006-source", but the DoFn's transform (DoOperation) where the timers are set. The latter doesn't know anything about splitting. Therefore, it just passes every single timers in the data channel. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
