[
https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279355#comment-16279355
]
Peng Cheng commented on SPARK-6473:
---
Looks like this issue reappear at some point:
getScalaVersion() will be called anyway even if not in dev/test mode:
https://github.com/apache/spark/blob/master/launcher/src/main/java/org/apache/spark/launcher/AbstractCommandBuilder.java#L196
It was suppressed simply because SPARK_SCALA_VERSION is always set by shell
script.
Should we add a test to ensure that it never happens?
> Launcher lib shouldn't try to figure out Scala version when not in dev mode
> ---
>
> Key: SPARK-6473
> URL: https://issues.apache.org/jira/browse/SPARK-6473
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
>Affects Versions: 1.4.0
>Reporter: Marcelo Vanzin
>Assignee: Marcelo Vanzin
>Priority: Minor
> Fix For: 1.4.0
>
>
> Thanks to [~nravi] for pointing this out.
> The launcher library currently always tries to figure out what's the build's
> scala version, even when it's not needed. That code is only used when setting
> some dev options, and relies on the layout of the build directories, so it
> doesn't work with the directory layout created by make-distribution.sh.
> Right now this works on Linux because bin/load-spark-env.sh sets the Scala
> version explicitly, but it would break the distribution on Windows, for
> example.
> Fix is pretty straight-forward.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
