[jira] [Commented] (TOREE-302) Respect SPARK_HOME environmental variable

[Dmytro Lituiev (JIRA)]

[ 
https://issues.apache.org/jira/browse/TOREE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969618#comment-15969618
 ] 

Dmytro Lituiev commented on TOREE-302:
--

Can this also be fixed in documentation?

https://toree.incubator.apache.org/documentation/user/installation.html

> Respect SPARK_HOME environmental variable
> -
>
> Key: TOREE-302
> URL: https://issues.apache.org/jira/browse/TOREE-302
> Project: TOREE
>  Issue Type: Improvement
>Affects Versions: 0.1.0
> Environment: OSX w/ bash
> Toree version 0.1.0.dev5 / commit 855fb4100b07
>Reporter: Seth Bromberger
>Assignee: Gino Bustelo
>Priority: Minor
> Fix For: 0.1.0
>
>
> I have SPARK_HOME set (because I installed spark via Homebrew). The toree 
> installation initially failed:
> ```
> phase:~ me$ jupyter toree install
> [ToreeInstall] Installing toree kernel with interpreter Scala
> [ToreeInstall] Removing existing kernelspec in 
> /usr/local/share/jupyter/kernels/toree_scala
> [ToreeInstall] Installed kernelspec toree_scala in 
> /usr/local/share/jupyter/kernels/toree_scala
> Traceback (most recent call last):
>   File "/Users/me/Library/Python/2.7/bin/jupyter-toree", line 9, in 
> load_entry_point('toree==0.1.0.dev5', 'console_scripts', 
> 'jupyter-toree')()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 149, in main
> ToreeApp.launch_instance()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/traitlets/config/application.py",
>  line 596, in launch_instance
> app.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 146, in start
> return self.subapp.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 115, in start
> self.create_kernel_json(install_dir, interpreter)
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 79, in create_kernel_json
> python_lib_contents = listdir("{0}/python/lib".format(self.spark_home))
> OSError: [Errno 2] No such file or directory: '/usr/local/spark/python/lib'
> ```
> I got this working by running `jupyter toree install 
> --spark_home=$SPARK_HOME`.
> It would be nice if SPARK_HOME were used automatically if it is set.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (TOREE-302) Respect SPARK_HOME environmental variable

[Gino Bustelo (JIRA)]

[ 
https://issues.apache.org/jira/browse/TOREE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264713#comment-15264713
 ] 

Gino Bustelo commented on TOREE-302:


fixed with https://github.com/apache/incubator-toree/pull/34

> Respect SPARK_HOME environmental variable
> -
>
> Key: TOREE-302
> URL: https://issues.apache.org/jira/browse/TOREE-302
> Project: TOREE
>  Issue Type: Improvement
>Affects Versions: 0.1.0
> Environment: OSX w/ bash
> Toree version 0.1.0.dev5 / commit 855fb4100b07
>Reporter: Seth Bromberger
>Assignee: Gino Bustelo
>Priority: Minor
> Fix For: 0.1.0
>
>
> I have SPARK_HOME set (because I installed spark via Homebrew). The toree 
> installation initially failed:
> ```
> phase:~ me$ jupyter toree install
> [ToreeInstall] Installing toree kernel with interpreter Scala
> [ToreeInstall] Removing existing kernelspec in 
> /usr/local/share/jupyter/kernels/toree_scala
> [ToreeInstall] Installed kernelspec toree_scala in 
> /usr/local/share/jupyter/kernels/toree_scala
> Traceback (most recent call last):
>   File "/Users/me/Library/Python/2.7/bin/jupyter-toree", line 9, in 
> load_entry_point('toree==0.1.0.dev5', 'console_scripts', 
> 'jupyter-toree')()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 149, in main
> ToreeApp.launch_instance()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/traitlets/config/application.py",
>  line 596, in launch_instance
> app.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 146, in start
> return self.subapp.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 115, in start
> self.create_kernel_json(install_dir, interpreter)
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 79, in create_kernel_json
> python_lib_contents = listdir("{0}/python/lib".format(self.spark_home))
> OSError: [Errno 2] No such file or directory: '/usr/local/spark/python/lib'
> ```
> I got this working by running `jupyter toree install 
> --spark_home=$SPARK_HOME`.
> It would be nice if SPARK_HOME were used automatically if it is set.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TOREE-302) Respect SPARK_HOME environmental variable

[Gino Bustelo (JIRA)]

[ 
https://issues.apache.org/jira/browse/TOREE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264687#comment-15264687
 ] 

Gino Bustelo commented on TOREE-302:


pairing with [~Lull3rSkat3r]

> Respect SPARK_HOME environmental variable
> -
>
> Key: TOREE-302
> URL: https://issues.apache.org/jira/browse/TOREE-302
> Project: TOREE
>  Issue Type: Improvement
>Affects Versions: 0.1.0
> Environment: OSX w/ bash
> Toree version 0.1.0.dev5 / commit 855fb4100b07
>Reporter: Seth Bromberger
>Assignee: Gino Bustelo
>Priority: Minor
> Fix For: 0.1.0
>
>
> I have SPARK_HOME set (because I installed spark via Homebrew). The toree 
> installation initially failed:
> ```
> phase:~ me$ jupyter toree install
> [ToreeInstall] Installing toree kernel with interpreter Scala
> [ToreeInstall] Removing existing kernelspec in 
> /usr/local/share/jupyter/kernels/toree_scala
> [ToreeInstall] Installed kernelspec toree_scala in 
> /usr/local/share/jupyter/kernels/toree_scala
> Traceback (most recent call last):
>   File "/Users/me/Library/Python/2.7/bin/jupyter-toree", line 9, in 
> load_entry_point('toree==0.1.0.dev5', 'console_scripts', 
> 'jupyter-toree')()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 149, in main
> ToreeApp.launch_instance()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/traitlets/config/application.py",
>  line 596, in launch_instance
> app.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 146, in start
> return self.subapp.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 115, in start
> self.create_kernel_json(install_dir, interpreter)
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 79, in create_kernel_json
> python_lib_contents = listdir("{0}/python/lib".format(self.spark_home))
> OSError: [Errno 2] No such file or directory: '/usr/local/spark/python/lib'
> ```
> I got this working by running `jupyter toree install 
> --spark_home=$SPARK_HOME`.
> It would be nice if SPARK_HOME were used automatically if it is set.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TOREE-302) Respect SPARK_HOME environmental variable

[Gino Bustelo (JIRA)]

[ 
https://issues.apache.org/jira/browse/TOREE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258660#comment-15258660
 ] 

Gino Bustelo commented on TOREE-302:


As discussed in gitter... we should have the install see the env_variable 
already set and use it.

[~Lull3rSkat3r]

> Respect SPARK_HOME environmental variable
> -
>
> Key: TOREE-302
> URL: https://issues.apache.org/jira/browse/TOREE-302
> Project: TOREE
>  Issue Type: Improvement
>Affects Versions: 0.1.0
> Environment: OSX w/ bash
> Toree version 0.1.0.dev5 / commit 855fb4100b07
>Reporter: Seth Bromberger
>Priority: Minor
> Fix For: 0.1.0
>
>
> I have SPARK_HOME set (because I installed spark via Homebrew). The toree 
> installation initially failed:
> ```
> phase:~ me$ jupyter toree install
> [ToreeInstall] Installing toree kernel with interpreter Scala
> [ToreeInstall] Removing existing kernelspec in 
> /usr/local/share/jupyter/kernels/toree_scala
> [ToreeInstall] Installed kernelspec toree_scala in 
> /usr/local/share/jupyter/kernels/toree_scala
> Traceback (most recent call last):
>   File "/Users/me/Library/Python/2.7/bin/jupyter-toree", line 9, in 
> load_entry_point('toree==0.1.0.dev5', 'console_scripts', 
> 'jupyter-toree')()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 149, in main
> ToreeApp.launch_instance()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/traitlets/config/application.py",
>  line 596, in launch_instance
> app.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 146, in start
> return self.subapp.start()
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 115, in start
> self.create_kernel_json(install_dir, interpreter)
>   File 
> "/Users/me/Library/Python/2.7/lib/python/site-packages/toree/toreeapp.py", 
> line 79, in create_kernel_json
> python_lib_contents = listdir("{0}/python/lib".format(self.spark_home))
> OSError: [Errno 2] No such file or directory: '/usr/local/spark/python/lib'
> ```
> I got this working by running `jupyter toree install 
> --spark_home=$SPARK_HOME`.
> It would be nice if SPARK_HOME were used automatically if it is set.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)