Repository: incubator-livy Updated Branches: refs/heads/master 54715447e -> 64d71aa75
[LIVY-418] Respect spark.pyspark.python for PythonInterpreter ## What changes were proposed in this pull request? `spark.pyspark.python` is introduced in spark 2.1, livy also need to update its code for choosing python version ## How was this patch tested? Manually verified. Author: Jeff Zhang <zjf...@gmail.com> Closes #65 from zjffdu/LIVY-418. Project: http://git-wip-us.apache.org/repos/asf/incubator-livy/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-livy/commit/64d71aa7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-livy/tree/64d71aa7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-livy/diff/64d71aa7 Branch: refs/heads/master Commit: 64d71aa757a93518427f315e908dcf54373948b2 Parents: 5471544 Author: Jeff Zhang <zjf...@gmail.com> Authored: Thu Nov 23 10:14:15 2017 +0800 Committer: jerryshao <ss...@hortonworks.com> Committed: Thu Nov 23 10:14:15 2017 +0800 ---------------------------------------------------------------------- repl/src/main/scala/org/apache/livy/repl/PythonInterpreter.scala | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-livy/blob/64d71aa7/repl/src/main/scala/org/apache/livy/repl/PythonInterpreter.scala ---------------------------------------------------------------------- diff --git a/repl/src/main/scala/org/apache/livy/repl/PythonInterpreter.scala b/repl/src/main/scala/org/apache/livy/repl/PythonInterpreter.scala index fa9be2d..f4f5e41 100644 --- a/repl/src/main/scala/org/apache/livy/repl/PythonInterpreter.scala +++ b/repl/src/main/scala/org/apache/livy/repl/PythonInterpreter.scala @@ -44,7 +44,8 @@ import org.apache.livy.sessions._ object PythonInterpreter extends Logging { def apply(conf: SparkConf, sparkEntries: SparkEntries): Interpreter = { - val pythonExec = sys.env.get("PYSPARK_PYTHON") + val pythonExec = conf.getOption("spark.pyspark.python") + .orElse(sys.env.get("PYSPARK_PYTHON")) .orElse(sys.props.get("pyspark.python")) // This java property is only used for internal UT. .getOrElse("python")
