tkaymak commented on issue #36841: URL: https://github.com/apache/beam/issues/36841#issuecomment-5639386451
Slice 4, the stateless streaming translator, merged as #40090. The Spark 4 runner now runs unbounded reads, stateless ParDo, Window.Assign, Flatten and Reshuffle in streaming mode, everything else fails at translation with a message pointing here. Found on the way: `cancel()` did not cancel Spark jobs and could stop a shared SparkSession, #40101, fix in #40103. Remaining, cut from #39576 in this order: state and timer internals, the transformWithState processor, the GroupByKey and stateful ParDo translators, then PAssert support. No CHANGES.md entry until windowed GroupByKey lands. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
