[
https://issues.apache.org/jira/browse/BEAM-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-7092:
-------------------------------
Description:
Since Spark 3 is relying on Scala 2.12 and a different set of dependencies
probably we need to divide the project and reuse most of the code as Flink
runner does to support multiple versions.
Probably the best (and simplest) path forward is to support Scala 2.11
Spark is deprecating Scala 2.11 support starting with the next release (3.0.0),
so it is probably a good idea to move the runner to use the already available
dependencies based on Scala 2.12.
was:
Since Spark 3 is relying on Scala 2.12
Spark is deprecating Scala 2.11 support starting with the next release (3.0.0),
so it is probably a good idea to move the runner to use the already available
dependencies based on Scala 2.12.
> Update Spark runner support multiple versions of Spark
> ------------------------------------------------------
>
> Key: BEAM-7092
> URL: https://issues.apache.org/jira/browse/BEAM-7092
> Project: Beam
> Issue Type: Sub-task
> Components: runner-spark
> Reporter: Ismaël Mejía
> Priority: Minor
>
> Since Spark 3 is relying on Scala 2.12 and a different set of dependencies
> probably we need to divide the project and reuse most of the code as Flink
> runner does to support multiple versions.
> Probably the best (and simplest) path forward is to support Scala 2.11
> Spark is deprecating Scala 2.11 support starting with the next release
> (3.0.0), so it is probably a good idea to move the runner to use the already
> available dependencies based on Scala 2.12.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
