[jira] [Commented] (LIVY-636) Unable to create interactive session with additional JAR in spark.driver.extraClassPath

[Konstantin (Jira)]

[ 
https://issues.apache.org/jira/browse/LIVY-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929104#comment-16929104
 ] 

Konstantin commented on LIVY-636:
-

If you're using sbt-assembly for making this JAR, there is an option:
{code:java}
assemblyOption in assembly := (assemblyOption in 
assembly).value.copy(includeScala = false)
{code}
Place this in build.sbt, and all should be fine.

> Unable to create interactive session with additional JAR in 
> spark.driver.extraClassPath
> ---
>
> Key: LIVY-636
> URL: https://issues.apache.org/jira/browse/LIVY-636
> Project: Livy
>  Issue Type: Bug
>Affects Versions: 0.6.0
>Reporter: Ishita Virmani
>Priority: Major
> Attachments: applicationmaster.log, container.log, stacktrace.txt, 
> test.png
>
>
> Command Run: c{{url -H "Content-Type: application/json" -X POST -d 
> '\{"kind":"pyspark","conf":{"spark.driver.extraClassPath":"/data/XXX-0.0.1-SNAPSHOT.jar"}}'
>  -i http:///session}}
> {{The above command fails to create a Spark Session on YARN with Null pointer 
> exception. Stack trace for the same has been attached along-with.}}
> The JAR file here is present on local driver Path. Also tried using HDFS path 
> in the following manner 
> {{hdfs://:/data/XXX-0.0.1-SNAPSHOT.jar}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (LIVY-636) Unable to create interactive session with additional JAR in spark.driver.extraClassPath

[Ishita Virmani (Jira)]

[ 
https://issues.apache.org/jira/browse/LIVY-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929079#comment-16929079
 ] 

Ishita Virmani commented on LIVY-636:
-

Hi [~Cerberuser],

Your solution seem to have worked. Thanks

But right now I manually removed Scala folder from my JAR. But is there any 
other way to remove Scala, since my library is coded in scala, and would thus 
include Scala in the JAR by default.

> Unable to create interactive session with additional JAR in 
> spark.driver.extraClassPath
> ---
>
> Key: LIVY-636
> URL: https://issues.apache.org/jira/browse/LIVY-636
> Project: Livy
>  Issue Type: Bug
>Affects Versions: 0.6.0
>Reporter: Ishita Virmani
>Priority: Major
> Attachments: applicationmaster.log, container.log, stacktrace.txt, 
> test.png
>
>
> Command Run: c{{url -H "Content-Type: application/json" -X POST -d 
> '\{"kind":"pyspark","conf":{"spark.driver.extraClassPath":"/data/XXX-0.0.1-SNAPSHOT.jar"}}'
>  -i http:///session}}
> {{The above command fails to create a Spark Session on YARN with Null pointer 
> exception. Stack trace for the same has been attached along-with.}}
> The JAR file here is present on local driver Path. Also tried using HDFS path 
> in the following manner 
> {{hdfs://:/data/XXX-0.0.1-SNAPSHOT.jar}}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)