[jira] [Resolved] (TOREE-344) No module named pyspark

[Luciano Resende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TOREE-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luciano Resende resolved TOREE-344.
---
   Resolution: Won't Fix
Fix Version/s: 0.2.0

This is true for any python dependencies and I believe its more like a runtime 
feature request rather than a kernel issue.

> No module named pyspark
> ---
>
> Key: TOREE-344
> URL: https://issues.apache.org/jira/browse/TOREE-344
> Project: TOREE
>  Issue Type: Bug
>Reporter: haniar
>Priority: Major
> Fix For: 0.2.0
>
>
> I have installed toree to my jupyter environment 
> (https://github.com/apache/incubator-toree) and written a piece of code that 
> works with pyspark. Yarn starts properly and I can see the containers running 
> in the queue,
> When I run the code, I get the following error
> Error from python worker:
>   /usr/local/bin/python2.7: No module named pyspark
> the kernel is set-up as follows:
> {
>   "language": "python",
>   "display_name": "Apache Toree - PySpark",
>   "env": {
> "__TOREE_SPARK_OPTS__": " --master yarn",
> "SPARK_HOME": "/usr/hdp/2.4.2.0-258/spark",
> "__TOREE_OPTS__": "",
> "DEFAULT_INTERPRETER": "PySpark",
> "PYTHONPATH": 
> "/usr/hdp/2.4.2.0-258/spark/python:/usr/hdp/2.4.2.0-258/spark/python/lib/py4j-0.9-src.zip",
> "PYTHON_EXEC": "python",
>  "PYTHONSTARTUP": "/usr/hdp/2.4.2.0-258/spark/python/pyspark/shell.py",
>  "PYSPARK_PYTHON": "/usr/local/bin/python2.7",
>"PYSPARK_DRIVER_PYTHON": "/usr/local/bin/python2.7"
>   },
>   "argv": [
> "/usr/local/share/jupyter/kernels/apache_toree_pyspark/bin/run.sh",
> "--profile",
> "{connection_file}"
>   ]
> }



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TOREE-399) Make Spark Kernel work on Windows

[Patrick Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/TOREE-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384251#comment-16384251
 ] 

Patrick Chan commented on TOREE-399:


I am completely new to this.   How can I set  -Dscala.usejavacp=true and 
include toree-assembly jar file in the classpath?    I didn't see these options 
in the kernel.json or run.cmd file.

> Make Spark Kernel work on Windows
> -
>
> Key: TOREE-399
> URL: https://issues.apache.org/jira/browse/TOREE-399
> Project: TOREE
>  Issue Type: New Feature
> Environment: Windows 7/8/10
>Reporter: aldo
>Priority: Major
> Attachments: run.bat
>
>
> After a successful install of the Spark Kernel the error: "Failed to run 
> command:" occurs when from jupyter we select a Scala Notebook.
> The error happens because the kernel.json runs 
> C:\\ProgramData\\jupyter\\kernels\\apache_toree_scala\\bin\\run.sh which is 
> bash shell script and hence cannot work on windows.
> Can you give me some direction to fix this, and I will implement it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)