[
https://issues.apache.org/jira/browse/TOREE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789841#comment-16789841
]
Luciano Resende commented on TOREE-495:
---------------------------------------
The current implementation takes anything added during the notebook runtime
(e.g. %addDeps and %addJar) as transient, and independent of the location it is
created it is created using the createTmp OS API that deletes the file after
the process is deleted. It seems that, what you need is just the ability to
have the user "dependencies" available in kernels, and you can do that by doing
the map as you mentioned together with updating the kernelspecto use additional
PYTHONPATH configuration (e.g. --conf
spark.yarn.appMasterEnv.PYTHONPATH=${HOME}/.local/lib/python2.7/site-packages:/usr/hdp/current/spark2-client/python:/usr/hdp/current/spark2-client/python/lib/py4j-0.10.6-src.zip)
the only caveat is that, permanent libraries would have to be added outside of
the notebook environment.
> Add deps-dir option to toree configuration options
> --------------------------------------------------
>
> Key: TOREE-495
> URL: https://issues.apache.org/jira/browse/TOREE-495
> Project: TOREE
> Issue Type: Improvement
> Reporter: Stanislav Kokaisl
> Priority: Major
>
> It would be helpful to have option, where are dependencies stored with
> AddDeps magic. Currently there is only {{jar-dir option for AddJar magic.}}
> {{We create new deps-dir attribute for toree_opts. This option can be then
> used in install command:}}
> {code:java}
> jupyter toree install --toree_opts="--deps-dir=/path/to/deps"{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
