If you're using 0.5.0-incubating, you can check "2 Loading Spark
Properties" in
http://zeppelin.incubator.apache.org/docs/interpreter/spark.html page.
Best,
moon
On Sat, Nov 14, 2015 at 4:37 PM Girish Reddy <gir...@springml.com> wrote:
> Does the spark.files option only work in 0.6? Any workaround for earlier
> versions to load external jars?
>
> On Tue, Oct 20, 2015 at 6:52 PM, moon soo Lee <m...@apache.org> wrote:
>
>> Sourav,
>>
>> There're couple of ways to add external jar when Zeppelin
>> (0.6.0-SNAPSHOT) uses spark-submit command.
>>
>> 1. Using %dep interpreter, as Vinay mentioned.
>> eg)
>> %dep
>> z.load("group:artifact:version")
>>
>> %spark
>> import ....
>>
>> 2. By adding spark.files property at SPARK_HOME/conf/spark-defaults.conf
>> eg)
>> spark.files /path/to/my.jar
>>
>> 3. By exporting SPARK_SUBMIT_OPTIONS env variable in
>> ZEPPELIN_HOME/conf/zeppelin-env.sh
>> eg)
>> export SPARK_SUBMIT_OPTIONS="--packages group:artifact:version"
>> note) does not work for pyspark yet.
>> https://issues.apache.org/jira/browse/ZEPPELIN-339
>>
>> Hope this helps.
>>
>> Best,
>> moon
>>
>> On Wed, Oct 21, 2015 at 12:55 AM Vinay Shukla <vinayshu...@gmail.com>
>> wrote:
>>
>>> Saurav,
>>>
>>>
>>> Agree this would be a useful feature. Right now Zeppelin can import
>>> dependencies from Maven with %dep interpreter. But this does not understand
>>> spark-packages.
>>>
>>>
>>> -Vinay
>>>
>>>
>>> On Tue, Oct 20, 2015 at 6:54 AM, Sourav Mazumder <
>>> sourav.mazumde...@gmail.com> wrote:
>>>
>>>> Looks like right now there is no way one can pass additional jar files
>>>> to spark_submit from Zeppelin.The same works fine if I am not using
>>>> spark_submit option (by not specifying spark_home).
>>>>
>>>> When I checked the code interpreter.sh I found that for the class path
>>>> it only passes the zeppelin-spark*.jar available in
>>>> zeppelin_home/interpreter/spark directory.
>>>>
>>>> I suggest to put this as a bug/enhancement. The solution should be
>>>> pretty easy by making some small changes in interpreter.sh (I've done the
>>>> same and could make it work with some external_lib folder under
>>>> zeppelin_home/interpreter/spark directory).
>>>>
>>>> Regards,
>>>> Sourav
>>>>
>>>
>>>
