[
https://issues.apache.org/jira/browse/TOREE-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gino Bustelo closed TOREE-310.
------------------------------
> Allow override of python executable used for pyspark
> ----------------------------------------------------
>
> Key: TOREE-310
> URL: https://issues.apache.org/jira/browse/TOREE-310
> Project: TOREE
> Issue Type: Improvement
> Reporter: Eric Chang
> Fix For: 0.1.0
>
>
> We're using virtualenvs when running pyspark, and it would be great to be
> able to use a virtualenv as the python executable used by the Spark Driver
> (i.e. {{--master yarn-client}}). This value is currently hard-coded to
> {{python}} in
> org/apache/toree/kernel/interpreter/pyspark/PySparkProcess.scala.
> I have a branch on my repo which adds an optional kernel parameter
> PYTHON_EXEC:
> {code}
> ...
> "SPARK_HOME": "/usr/lib/spark",
> "PYTHON_EXEC" : "/usr/local/python/virtualenvs/myvenv/bin/python",
> ...
> {code}
> If {{PYTHON_EXEC}} is unspecified, the default of {{python}} is used.
> Here's the diff of the branch, please let me know if it's ok for me to issue
> a PR against the main repo:
> https://github.com/ericchang/incubator-toree/compare/ericchang:master...custom-python-exec
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
