[
https://issues.apache.org/jira/browse/BEAM-9933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17392273#comment-17392273
]
Etienne Chauchot commented on BEAM-9933:
----------------------------------------
This is due to the watermark not properly implemented in spark structured
streaming framework (apache spark side) : see
https://echauchot.blogspot.com/2020/11/watermark-architecture-proposal-for.html
> Support Streaming in Spark Structured Streaming Runner
> ------------------------------------------------------
>
> Key: BEAM-9933
> URL: https://issues.apache.org/jira/browse/BEAM-9933
> Project: Beam
> Issue Type: New Feature
> Components: runner-spark
> Reporter: Ismaël Mejía
> Priority: P3
> Labels: structured-streaming
>
> The current version of the Spark Structured Streaming Runner only support
> Bounded IO. This ticket is to address the support of full Streaming.
> This issue is blocked because of issues on Spark with multiple aggregations
> SPARK-26655
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
