Hi, I was actually trying to integrate Flume->Spark->Zeppelin. I used the %dep dependency loader to load the compiled packages, like you mentioned. Then I invoked the main class from my package by defining an instance variable on the REPL using %spark interpreter. It worked partially proving that such invocation is possible.
But in my use case, Flume needs the port number of the Spark master. Since the REPL does not use a port, I got an exception (connect exception). Regards, Prajod From: moon soo Lee [mailto:[email protected]] Sent: 28 April 2015 04:55 To: [email protected] Subject: Re: spark submit to zeppelin built in Local mode Hi Prajod, Zeppelin uses scala REPL in it's SparkInterpreter and scala REPL not allowing defining a package. To submit existing code, http://zeppelin.incubator.apache.org/docs/interpreter/spark.html#dependencyloading might help. Is it helping you? Thanks, moon On Tue, Apr 28, 2015 at 2:34 AM <[email protected]<mailto:[email protected]>> wrote: Hi, Is it possible to do a spark_submit with zeppelin built in Local mode ? The Notebook accepts only code snippets. So that is not an alternative. Eg: The Notebook does not allow package names. A line like: package com.mycompany.settlement results in a parse error in the Notebook. If it is possible to submit existing code to the Local mode’s spark interpreter, that would be very useful for developers, by avoiding the need to run zeppelin and spark as separate servers. Thanks, Prajod The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com<http://www.wipro.com> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
