kennknowles commented on PR #28272: URL: https://github.com/apache/beam/pull/28272#issuecomment-1725760352
Yea let's sync on this. I think I've come around but I want to get it on dev@ in at least a 1 pager for what the spec is. If it turns out to not be dangerous to reinstate, and we can regain the semantic property "reshuffle is a semantic noop" then that is pretty desirable, so that runners can freely add and remove reshuffles. Noting that here is where Dataflow v1 implemented reshuffle and re-instated the pre-reshuffle pane: https://github.com/apache/beam/blob/34024902746af90e7bf41e28729ec031dbab58d2/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingGroupAlsoByWindowReshuffleFn.java#L61 Noting that Beam's reference implementation, the closest thing we have to a spec, does not preserve the pane but will produce a sequence of increasing panes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
