reuvenlax commented on PR #28272: URL: https://github.com/apache/beam/pull/28272#issuecomment-1716337512
I'm not sure what you mean by random upstream keys. Reshuffle currently maps KV -> KV, so no keys are changed. The only difference is that the keys are now spread out on different workers, which is a semantic no op. You may be thinking of Rehsuffle.viaRandomKey, which is a bit different. On Tue, Sep 12, 2023 at 6:33 AM Kenn Knowles ***@***.***> wrote: > Actually each GBK has its own sequence of trigger firings, and when it > comes to reshuffle there are two possibilities: > > 1. The reshuffle does a GBK then explodes it (per its expansion) in > which case the sequence of pane indices come from the Reshuffle/always > trigger on that GBK. > 2. The reshuffle is a semantics "no op" at the PCollection level, in > which case the pane indices would be preserved from an upstream > aggregation, but reshuffle results in bundles from random upstream keys so > they are deceptive. > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/beam/pull/28272#issuecomment-1715736420>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AFAYJVLDCEOGYBQCRHRTBSDX2BQDFANCNFSM6AAAAAA4GWNMFI> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
