[
https://issues.apache.org/jira/browse/BEAM-9933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102734#comment-17102734
]
Ismaël Mejía commented on BEAM-9933:
------------------------------------
Spark has a checker of unsupported operations that matches this
https://github.com/apache/spark/blob/0fb607ef3744e073a09423460a79e3325b24b6ad/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala#L128
It can be somehow avoided by workaround it with bytecode manipulation but this
is not an ideal solution.
> Support Streaming in Spark Structured Streaming Runner
> ------------------------------------------------------
>
> Key: BEAM-9933
> URL: https://issues.apache.org/jira/browse/BEAM-9933
> Project: Beam
> Issue Type: New Feature
> Components: runner-spark
> Reporter: Ismaël Mejía
> Priority: Major
> Labels: structured-streaming
>
> The current version of the Spark Structured Streaming Runner only support
> Bounded IO. This ticket is to address the support of full Streaming.
> This issue is blocked because of issues on Spark with multiple aggregations
> SPARK-26655
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
