Eliaaazzz commented on PR #39576:
URL: https://github.com/apache/beam/pull/39576#issuecomment-5217598039
Hi @tkaymak, I found this PR while looking into Spark streaming work.
I had been planning to work on Spark portable streaming state and side
inputs via #20396 and
#20395, but it looks like new streaming work is moving toward the Spark 4
Structured Streaming
runner rather than the older DStream path.
I’m reading through this POC now so I do not duplicate work or build
against the wrong runner
path. If this approach is still the intended direction, is there a smaller
slice where outside
help would be useful?
My initial interest was state/side-input support, but I see this POC
already covers stateful
ParDo and deliberately leaves streaming side inputs out of scope. I’d be
happy to help with
tests, extracting one of the proposed smaller PRs, or another piece that
would be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
