[jira] [Commented] (TOREE-371) $SPARK_HOME environment variable not recognised
[
https://issues.apache.org/jira/browse/TOREE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942670#comment-15942670
]
Junyoung Park commented on TOREE-371:
-
I had the same issue. The default folder path is incorrect and can not be read.
I fixed it by specifying SPARK_HOME variable.
If installed by Homebrew, users should be export env variables.
export SPARK_HOME=/usr/local/Cellar/apache-spark/2.1.0/libexec
> $SPARK_HOME environment variable not recognised
> ---
>
> Key: TOREE-371
> URL: https://issues.apache.org/jira/browse/TOREE-371
> Project: TOREE
> Issue Type: Bug
>Affects Versions: 0.1.0
> Environment: 64 bit Ubuntu 16.04 LTS
>Reporter: Peter
>
> Hi
> toreeapp.py is not recognising $SPARK_HOME, which is exported in .bashrc.
> toreeapp.py is instead defaulting location of spark.
> $SPARK_HOME is read on line 57 in toree.py as follows:
> spark_home = Unicode(os.getenv(SPARK_HOME, '/usr/local/spark')
> $ echo $SPARK_HOME
> /usr/lib/spark
> $ sudo jupyter toree install --spark_opts='--master=local[4]'
> [ToreeInstall] Installing Apache Toree version 0.1.0.dev8
> [ToreeInstall]
> Apache Toree is an effort undergoing incubation at the Apache Software
> Foundation (ASF), sponsored by the Apache Incubator PMC.
> Incubation is required of all newly accepted projects until a further review
> indicates that the infrastructure, communications, and decision making process
> have stabilized in a manner consistent with other successful ASF projects.
> While incubation status is not necessarily a reflection of the completeness
> or stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.
> Additionally, this release is not fully compliant with Apache release policy
> and includes a runtime dependency that is licensed as LGPL v3 (plus a static
> linking exception). This package is currently under an effort to re-license
> (https://github.com/zeromq/jeromq/issues/327).
> [ToreeInstall] Creating kernel Scala
> [ToreeInstall] Removing existing kernelspec in
> /usr/local/share/jupyter/kernels/apache_toree_scala
> [ToreeInstall] Installed kernelspec apache_toree_scala in
> /usr/local/share/jupyter/kernels/apache_toree_scala
> Traceback (most recent call last):
> File "/usr/local/bin/jupyter-toree", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 167,
> in main
> ToreeApp.launch_instance()
> File
> "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py",
> line 653, in launch_instance
> app.start()
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 164,
> in start
> return self.subapp.start()
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 133,
> in start
> self.create_kernel_json(install_dir, interpreter)
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 90,
> in create_kernel_json
> python_lib_contents = listdir("{0}/python/lib".format(self.spark_home))
> FileNotFoundError: [Errno 2] No such file or directory:
> '/usr/local/spark/python/lib'
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (TOREE-371) $SPARK_HOME environment variable not recognised
[
https://issues.apache.org/jira/browse/TOREE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854859#comment-15854859
]
Jakob Odersky commented on TOREE-371:
-
I'm not sure how changing the default value fixes the issue, however I'm glad
it worked for you. Would you like to open a pull request?
> $SPARK_HOME environment variable not recognised
> ---
>
> Key: TOREE-371
> URL: https://issues.apache.org/jira/browse/TOREE-371
> Project: TOREE
> Issue Type: Bug
>Affects Versions: 0.1.0
> Environment: 64 bit Ubuntu 16.04 LTS
>Reporter: Peter
>
> Hi
> toreeapp.py is not recognising $SPARK_HOME, which is exported in .bashrc.
> toreeapp.py is instead defaulting location of spark.
> $SPARK_HOME is read on line 57 in toree.py as follows:
> spark_home = Unicode(os.getenv(SPARK_HOME, '/usr/local/spark')
> $ echo $SPARK_HOME
> /usr/lib/spark
> $ sudo jupyter toree install --spark_opts='--master=local[4]'
> [ToreeInstall] Installing Apache Toree version 0.1.0.dev8
> [ToreeInstall]
> Apache Toree is an effort undergoing incubation at the Apache Software
> Foundation (ASF), sponsored by the Apache Incubator PMC.
> Incubation is required of all newly accepted projects until a further review
> indicates that the infrastructure, communications, and decision making process
> have stabilized in a manner consistent with other successful ASF projects.
> While incubation status is not necessarily a reflection of the completeness
> or stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.
> Additionally, this release is not fully compliant with Apache release policy
> and includes a runtime dependency that is licensed as LGPL v3 (plus a static
> linking exception). This package is currently under an effort to re-license
> (https://github.com/zeromq/jeromq/issues/327).
> [ToreeInstall] Creating kernel Scala
> [ToreeInstall] Removing existing kernelspec in
> /usr/local/share/jupyter/kernels/apache_toree_scala
> [ToreeInstall] Installed kernelspec apache_toree_scala in
> /usr/local/share/jupyter/kernels/apache_toree_scala
> Traceback (most recent call last):
> File "/usr/local/bin/jupyter-toree", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 167,
> in main
> ToreeApp.launch_instance()
> File
> "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py",
> line 653, in launch_instance
> app.start()
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 164,
> in start
> return self.subapp.start()
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 133,
> in start
> self.create_kernel_json(install_dir, interpreter)
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 90,
> in create_kernel_json
> python_lib_contents = listdir("{0}/python/lib".format(self.spark_home))
> FileNotFoundError: [Errno 2] No such file or directory:
> '/usr/local/spark/python/lib'
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (TOREE-371) $SPARK_HOME environment variable not recognised
[
https://issues.apache.org/jira/browse/TOREE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15853966#comment-15853966
]
Peter commented on TOREE-371:
-
I modified line 57 in /usr/local/lib/python3.5/dist-packages/toree/toreeapp.py
to the following:
spark_home = Unicode(os.getenv(SPARK_HOME, '/usr/local/bin/apache-spark'),
config=True,
since "/usr/local/bin/apache-spark" is the default directory selected by the
script.
I used $ sudo vi toreeapp.py to edit the file.
Everything installed fine after that.
> $SPARK_HOME environment variable not recognised
> ---
>
> Key: TOREE-371
> URL: https://issues.apache.org/jira/browse/TOREE-371
> Project: TOREE
> Issue Type: Bug
>Affects Versions: 0.1.0
> Environment: 64 bit Ubuntu 16.04 LTS
>Reporter: Peter
>
> Hi
> toreeapp.py is not recognising $SPARK_HOME, which is exported in .bashrc.
> toreeapp.py is instead defaulting location of spark.
> $SPARK_HOME is read on line 57 in toree.py as follows:
> spark_home = Unicode(os.getenv(SPARK_HOME, '/usr/local/spark')
> $ echo $SPARK_HOME
> /usr/lib/spark
> $ sudo jupyter toree install --spark_opts='--master=local[4]'
> [ToreeInstall] Installing Apache Toree version 0.1.0.dev8
> [ToreeInstall]
> Apache Toree is an effort undergoing incubation at the Apache Software
> Foundation (ASF), sponsored by the Apache Incubator PMC.
> Incubation is required of all newly accepted projects until a further review
> indicates that the infrastructure, communications, and decision making process
> have stabilized in a manner consistent with other successful ASF projects.
> While incubation status is not necessarily a reflection of the completeness
> or stability of the code, it does indicate that the project has yet to be
> fully endorsed by the ASF.
> Additionally, this release is not fully compliant with Apache release policy
> and includes a runtime dependency that is licensed as LGPL v3 (plus a static
> linking exception). This package is currently under an effort to re-license
> (https://github.com/zeromq/jeromq/issues/327).
> [ToreeInstall] Creating kernel Scala
> [ToreeInstall] Removing existing kernelspec in
> /usr/local/share/jupyter/kernels/apache_toree_scala
> [ToreeInstall] Installed kernelspec apache_toree_scala in
> /usr/local/share/jupyter/kernels/apache_toree_scala
> Traceback (most recent call last):
> File "/usr/local/bin/jupyter-toree", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 167,
> in main
> ToreeApp.launch_instance()
> File
> "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py",
> line 653, in launch_instance
> app.start()
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 164,
> in start
> return self.subapp.start()
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 133,
> in start
> self.create_kernel_json(install_dir, interpreter)
> File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 90,
> in create_kernel_json
> python_lib_contents = listdir("{0}/python/lib".format(self.spark_home))
> FileNotFoundError: [Errno 2] No such file or directory:
> '/usr/local/spark/python/lib'
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
