[
https://issues.apache.org/jira/browse/TOREE-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luciano Resende resolved TOREE-344.
-----------------------------------
Resolution: Won't Fix
Fix Version/s: 0.2.0
This is true for any python dependencies and I believe its more like a runtime
feature request rather than a kernel issue.
> No module named pyspark
> -----------------------
>
> Key: TOREE-344
> URL: https://issues.apache.org/jira/browse/TOREE-344
> Project: TOREE
> Issue Type: Bug
> Reporter: haniar
> Priority: Major
> Fix For: 0.2.0
>
>
> I have installed toree to my jupyter environment
> (https://github.com/apache/incubator-toree) and written a piece of code that
> works with pyspark. Yarn starts properly and I can see the containers running
> in the queue,
> When I run the code, I get the following error
> Error from python worker:
> /usr/local/bin/python2.7: No module named pyspark
> the kernel is set-up as follows:
> {
> "language": "python",
> "display_name": "Apache Toree - PySpark",
> "env": {
> "__TOREE_SPARK_OPTS__": " --master yarn",
> "SPARK_HOME": "/usr/hdp/2.4.2.0-258/spark",
> "__TOREE_OPTS__": "",
> "DEFAULT_INTERPRETER": "PySpark",
> "PYTHONPATH":
> "/usr/hdp/2.4.2.0-258/spark/python:/usr/hdp/2.4.2.0-258/spark/python/lib/py4j-0.9-src.zip",
> "PYTHON_EXEC": "python",
> "PYTHONSTARTUP": "/usr/hdp/2.4.2.0-258/spark/python/pyspark/shell.py",
> "PYSPARK_PYTHON": "/usr/local/bin/python2.7",
> "PYSPARK_DRIVER_PYTHON": "/usr/local/bin/python2.7"
> },
> "argv": [
> "/usr/local/share/jupyter/kernels/apache_toree_pyspark/bin/run.sh",
> "--profile",
> "{connection_file}"
> ]
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
