kennknowles commented on issue #28219: URL: https://github.com/apache/beam/issues/28219#issuecomment-1714118666
I think there may be too much trial-and-error guessing here. We can reason about this and get it right. When you have an aggregation (either a GBK or a Combine) then the elements coming out of that aggregation are uniquely identified by key + window + pane index. The pane index is how you can tell the difference between different triggerings. A "Reshuffle" has a trigger that always fires as fast as possible. The pane index should increase with each element. But if it is by random key then the actual key that was shuffled by is dropped so the pane index doesn't mean anything any more. Now the problem is that Reshuffle also breaks apart the GBK result into individual elements again. There is no guarantee that these elements will always travel together in a bundle, so again the pane index is not a reliable thing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
