[
https://issues.apache.org/jira/browse/TOREE-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262719#comment-15262719
]
Corey A Stubbs commented on TOREE-308:
--------------------------------------
[~lbustelo], we are using the java classpath to search for our jars
[here|https://github.com/apache/incubator-toree/blob/master/plugins/src/main/scala/org/apache/toree/plugins/PluginSearcher.scala#L104].
The jar needs to be added to the `--driver-class-path` for it to appear in the
java classpath.
To fix this without adding another environment variable would entail one of the
following:
1. Figure out if spark submit is adding that jar to some other classpath and
use that classpath instead
2. Passing in our jar via an argument and adding it to the search classpath
above
A fix where we leave the command line arg would entail detecting it in
SPARK_OPTS and merging it with the --driver-class-path arg we have in run.sh
> Interpreter jar not found
> -------------------------
>
> Key: TOREE-308
> URL: https://issues.apache.org/jira/browse/TOREE-308
> Project: TOREE
> Issue Type: Bug
> Affects Versions: 0.1.0
> Reporter: Phil Berkland
> Fix For: 0.1.0
>
>
> We previously added our interpreter plugin by adding
> "SPARK_OPTS": "--jars file:/path/to/our.jar",
> to kernel.json.
> This no longer works, we get a message that the interpreter was not found.
> We discovered if we modified the --driver-class-path in toree/run.sh to
> include our jar, the interpreter was found.
> SPARK_OPTS="--driver-class-path=\"${TOREE_ASSEMBLY}:/path/to/our.jar\"
> ${SPARK_OPTS}"
> Unfortunately, this cannot be specified in the kernel.json, because the
> TOREE_ASSEMBLY would be lost.
> A new ENV var could be added (i.e. TOREE_DRIVER_JAR)
>
> SPARK_OPTS="--driver-class-path=\"${TOREE_ASSEMBLY}:${TOREE_DRIVER_JAR}:\"
> ${SPARK_OPTS}"
> which we could specifiy in our kernel.json
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
