Hello
I could fix this issue. It was only the problem of the URL of the file. I
was using like below
z.load("file:///home/bala/mapreduce.jar")
But, when I omitted file:// from the above url, it worked
z.load("/home/bala/mapreduce.jar")
regards
Bala
On 26 January 2016 at 08:23, Balachandar R.A. <balachandar...@gmail.com>
wrote:
> Hi falmeida,
>
> Thanks for the response but I do not want to use
> SPARK-SUBMIT-OPTIONS
>
> thanks and regards
> Bala
> On 26-Jan-2016 8:05 am, "Felipe Almeida" <falmeida1...@gmail.com> wrote:
>
>> Hi Balachandar I think I just went through that very same problem and I
>> solved it with the help of *Moon Soo Lee:*
>>
>> Here is the solution:
>> http://stackoverflow.com/questions/35005455/java-npe-when-loading-a-dependency-from-maven-from-within-zeppelin-on-aws-emr
>>
>> On 25 January 2016 at 14:27, Hyung Sung Shim <hss...@nflabs.com> wrote:
>>
>>> Hello.
>>> I also don't work the loadAndDist() method. Maybe it's deprecated.
>>> I'll check and fix the documents.
>>>
>>> using spark-shell, you can run your application as following step.
>>> 1. remove specified jar configuration in the spark-defaults.conf
>>> 2. in the spark-home, bin/spark-shell* --jars "YOUR JAR COMMA SPERATED"
>>> *
>>>
>>> If you share your application code and environments informations(zeppelin
>>> and spark version you're using, and zeppelin-env.sh etc..), i might help
>>> you.
>>>
>>> Thanks.
>>>
>>> <https://gitlab.com/search?group_id=&project_id=769187&scope=issues&search=spark-shell#2-function-define>
>>>
>>>
>>>
>>>
>>> 2016-01-25 19:08 GMT+09:00 Balachandar R.A. <balachandar...@gmail.com>:
>>>
>>>> Hello,
>>>>
>>>> I tried to use z.loadAndDist() but it says
>>>>
>>>> console>:17: error: value loadAndDist is not a member of
>>>> org.apache.zeppelin.spark.dep.DependencyContext
>>>>
>>>> Any idea here what this method is for?
>>>>
>>>>
>>>> regards
>>>> Bala
>>>>
>>>> On 25 January 2016 at 15:34, Balachandar R.A. <balachandar...@gmail.com
>>>> > wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I have run the code in spark-shell successfully but the jar files were
>>>>> all specified in the config files (spark-defaults.conf). However, I will
>>>>> not be able to use z.load() in spark-shell. Isn't? I am sorry but I did
>>>>> not
>>>>> pick up the idea of running using spark-shell. Wail suggestion is to
>>>>> create
>>>>> a fatJar? I will give it a try but still how do i make sure this fatJar is
>>>>> accessible to spark executors? ANyway, I will keep you posted on this
>>>>>
>>>>> regards
>>>>> Bala
>>>>>
>>>>> On 25 January 2016 at 13:39, Hyung Sung Shim <hss...@nflabs.com>
>>>>> wrote:
>>>>>
>>>>>> Hello.
>>>>>> I think Wail Alkowaileet's comment is possible.
>>>>>> Balachandar, Could you try to run your application with spark-shell?
>>>>>>
>>>>>>
>>>>>> 2016-01-25 15:45 GMT+09:00 Wail Alkowaileet <wael....@gmail.com>:
>>>>>>
>>>>>>> I used z.load in my case and it seems to be working just fine.
>>>>>>> Can you try spark-shell with your jar file? and see what is the
>>>>>>> error?
>>>>>>>
>>>>>>> I assume the problem that your application requires third-party
>>>>>>> jars. Therefore, you need to build your app with 'assembly'.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jan 25, 2016 at 9:39 AM, Balachandar R.A. <
>>>>>>> balachandar...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello Hyung,
>>>>>>>>
>>>>>>>> There is nothig I could make out from error log as it is plain
>>>>>>>> straightforward that classNotFoundException
>>>>>>>>
>>>>>>>> On 25 January 2016 at 11:34, Hyung Sung Shim <hss...@nflabs.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> It's weird..so Could you send the error log for details?
>>>>>>>>>
>>>>>>>>> 2016-01-25 15:00 GMT+09:00 Balachandar R.A. <
>>>>>>>>> balachandar...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi Hyung,
>>>>>>>>>>
>>>>>>>>>> Thanks for the response. This I have tried but did not work.
>>>>>>>>>>
>>>>>>>>>> regards
>>>>>>>>>> Bala
>>>>>>>>>>
>>>>>>>>>> On 25 January 2016 at 11:27, Hyung Sung Shim <hss...@nflabs.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello. Balachandar.
>>>>>>>>>>> In case of third one that you've tried, It must be first
>>>>>>>>>>> executed in the notebook.
>>>>>>>>>>> Could you try restart the zeppelin and run first the "%dep
>>>>>>>>>>> z.load()" paragraph?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2016-01-25 14:39 GMT+09:00 Balachandar R.A. <
>>>>>>>>>>> balachandar...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>> Any help would be greatly appreciated :-)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>>>>> From: Balachandar R.A. <balachandar...@gmail.com>
>>>>>>>>>>>> Date: 21 January 2016 at 14:11
>>>>>>>>>>>> Subject: Providing third party jar files to spark
>>>>>>>>>>>> To: users@zeppelin.incubator.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hello
>>>>>>>>>>>>
>>>>>>>>>>>> My spark based map tasks needs to access third party jar files.
>>>>>>>>>>>> I found below options to submit third party jar files to spark
>>>>>>>>>>>> interpreter
>>>>>>>>>>>>
>>>>>>>>>>>> 1. export SPARK_SUBMIT_OPTIONS=<all the jar files with comma
>>>>>>>>>>>> seprated> in conf/zeppelin-env.sh
>>>>>>>>>>>>
>>>>>>>>>>>> 2. include the statement spark.jars <all the jar files with
>>>>>>>>>>>> comma separated> in <spark>?conf/spark-defaults.conf
>>>>>>>>>>>>
>>>>>>>>>>>> 3. use the z.load("the location of jar file in the local
>>>>>>>>>>>> filesystem") in zepelin notebook
>>>>>>>>>>>>
>>>>>>>>>>>> I could test the first two and they both works fine. The third
>>>>>>>>>>>> one does not work. Here is the snippet i use
>>>>>>>>>>>>
>>>>>>>>>>>> %dep
>>>>>>>>>>>> z.reset()
>>>>>>>>>>>>
>>>>>>>>>>>> z.load("file:///home/bala/Projects/pocv8.new/mapreduce/build/libs/mapreduce.jar")
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Further, the import of class belongs to the above jar file is
>>>>>>>>>>>> working when I use the statement import com..... in zeppelin
>>>>>>>>>>>> notebook.
>>>>>>>>>>>> However, I get the class not found exception in the executor for
>>>>>>>>>>>> the same
>>>>>>>>>>>> class.
>>>>>>>>>>>>
>>>>>>>>>>>> Any clue here would help greatly
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> regards
>>>>>>>>>>>> Bala
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Regards,*
>>>>>>> Wail Alkowaileet
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> “Every time you stay out late; every time you sleep in; every time you
>> miss a workout; every time you don’t give 100% – You make it that much
>> easier for me to beat you.” - Unknown author
>>
>
