Josh Rosen created SPARK-3734:
---------------------------------
Summary: DriverRunner should not read SPARK_HOME from submitter's
environment
Key: SPARK-3734
URL: https://issues.apache.org/jira/browse/SPARK-3734
Project: Spark
Issue Type: Bug
Components: Deploy
Affects Versions: 1.1.0, 1.2.0
Reporter: Josh Rosen
Assignee: Josh Rosen
If you use {{spark-submit}} in {{cluster}} mode to submit a job to a Spark
Standalone cluster, and the {{JAVA_HOME}} environment variable is set on the
submitting machine, then DriverRunner will attempt to use the _submitter's_
JAVA_HOME to launch the driver process (instead of the worker's JAVA_HOME),
which can cause the job to fail unless the submitter and worker don't have Java
installed in the same location.
This has a pretty simple fix: read JAVA_HOME from {{sys.env}} instead of
{{command.environment}}; PR pending shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
