Mathew created ZEPPELIN-3720:
--------------------------------
Summary: PYSPARK_PYTHON / PYSPARK_DRIVER_PYTHON ignored.
Key: ZEPPELIN-3720
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3720
Project: Zeppelin
Issue Type: Bug
Affects Versions: 0.8.0
Environment: Spark 2.2.2 (Cloudera)
Zeppelin 0.8.0
RHEL 7.4
Reporter: Mathew
I am not 100% sure if the issue is on my end, but no matter what combination of
the following spark interpreter configs I use, the PYSPARK_PYTHON present in my
spark-env.sh takes precedence.
* PYSPARK_PYTHON
* PYSPARK_DRIVER_PYTHON
* spark.pyspark.python
* spark.pyspark.driver.python
* spark.yarn.appMasterEnv.PYSPARK_PYTHON
* spark.yarn.appMasterEnv.PYSPARK_DRIVER_PYTHON
Note that this is only is yarn client mode, as in yarn cluster mode, it ignores
whats in spark-env.sh.
When I use spark-submit it works fine, and picks up my custom python executable
path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
