reuvenlax commented on PR #28272: URL: https://github.com/apache/beam/pull/28272#issuecomment-1722574076
FYI - https://issues.apache.org/jira/browse/BEAM-7195 was caused by the inverse issue. The old code assumed that Reshuffle would produce pane indices, but of course it just forwards them. Let's wait until we figure out the correct behavior. Very few users will be exercising this from runner v2 today, so I would rather be cautious here - especially if the right answer is to fix runner v2. On Fri, Sep 15, 2023 at 3:33 AM Ahmed Abualsaud ***@***.***> wrote: > @reuvenlax <https://github.com/reuvenlax> as @Abacn > <https://github.com/Abacn> mentioned, we broke update-compatibility > recently in #28312 <https://github.com/apache/beam/pull/28312> so that we > can unblock dynamic destinations with copy jobs (#28309 > <https://github.com/apache/beam/issues/28309>). Do we have the green > light to merge these changes? > If Dataflow folks decide Runner V2 should preserve pane indices, that > support may still take some time to implement. In the meantime, this > ensures file loads streaming works on the runner (IIUC it previously never > did) > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/beam/pull/28272#issuecomment-1721050326>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AFAYJVINXQC4WCJC6KPYJB3X2QVHVANCNFSM6AAAAAA4GWNMFI> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
