[
https://issues.apache.org/jira/browse/BEAM-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beam JIRA Bot updated BEAM-9451:
--------------------------------
Priority: P3 (was: P2)
> Optimize translation when Schema information is available in Spark Structured
> Streaming runner
> ----------------------------------------------------------------------------------------------
>
> Key: BEAM-9451
> URL: https://issues.apache.org/jira/browse/BEAM-9451
> Project: Beam
> Issue Type: Improvement
> Components: runner-spark
> Reporter: Ismaël Mejía
> Priority: P3
> Labels: stale-P2, structured-streaming
>
> Spark Structured Streaming runner supports Datasets that already have Schema
> information. This is used by Spark to optimize jobs (via Catalyst). This
> issue is to implement optimized translations of the transforms for the runner
> so we can benefit of the performance improvements internally done by Spark.
> Notice that we also may need to map Beam's core internal representations like
> WindowedValue so we can have intermediary optimizations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
