tkaymak opened a new issue, #40949: URL: https://github.com/apache/beam/issues/40949
### What needs to happen? The Spark 4 Structured Streaming runner rejects a ParDo with additional output tags at translation, `PipelineTranslatorStreaming` throws "ParDo with additional outputs is not supported". The batch translator supports them. Transforms that expand to such a ParDo fail the same way, for example `Combine.perKey().withHotKeyFanout`, which splits hot and cold keys with `withOutputTags`. This keeps 17 ValidatesRunner tests out of the streaming suite `validatesStructuredStreamingRunnerStreaming`, among them `ParDoTest$MultipleInputsAndOutputTests`, `MetricsTest$AttemptedMetricTests`, `CombineTest$BasicTests.testHotKeyCombining` and `PCollectionTupleTest.testComposePCollectionTuple`. They are excluded by name there and cite this issue. Removing those exclusions is the acceptance check. Part of #36841. ### Issue Priority Priority: 3 (nice-to-have improvement) ### Issue Components - [x] Component: Spark Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
