Re: Zeppelin - Spark Driver location

spark-submit would only run when you run the first paragraph using spark interpreter. After that, paragraph would send code to the spark app to execute. >>> Also spark standalone cluster moder should work even before this new release, right? I didn't verify that, not sure whether other people

Re: Zeppelin - Spark Driver location

Also spark standalone cluster moder should work even before this new release, right? On Wed, Mar 14, 2018 at 8:43 AM, ankit jain wrote: > Hi Jhang, > Not clear on that - I thought spark-submit was done when we run a > paragraph, how does the .sh file come into play? > >

Re: Zeppelin - Spark Driver location

Hi Jhang, Not clear on that - I thought spark-submit was done when we run a paragraph, how does the .sh file come into play? Thanks Ankit On Tue, Mar 13, 2018 at 5:43 PM, Jeff Zhang wrote: > > spark-submit is called in bin/interpreter.sh, I didn't try standalone > cluster

Re: Zeppelin - Spark Driver location

spark-submit is called in bin/interpreter.sh, I didn't try standalone cluster mode. It is expected to run driver in separate host, but didn't guaranteed zeppelin support this. Ankit Jain 于2018年3月14日周三 上午8:34写道： > Hi Jhang, > What is the expected behavior with standalone

Re: Zeppelin - Spark Driver location

Hi Jhang, What is the expected behavior with standalone cluster mode? Should we see separate driver processes in the cluster(one per user) or multiple SparkSubmit processes? I was trying to dig in Zeppelin code & didn’t see where Zeppelin does the Spark-submit to the cluster? Can you please

Re: Zeppelin - Spark Driver location

ZEPPELIN-2898 is for yarn cluster model. And Zeppelin have integration test for yarn mode, so guaranteed it would work. But don't' have test for standalone, so not sure the behavior of standalone mode. Ruslan Dautkhanov

Re: Zeppelin - Spark Driver location

https://github.com/apache/zeppelin/pull/2577 pronounces yarn-cluster in it's title so I assume it's only yarn-cluster. Never used standalone-cluster myself. Which distro of Hadoop do you use? Cloudera desupported standalone in CDH 5.5 and will remove in CDH 6.

Re: Zeppelin - Spark Driver location

Does this new feature work only for yarn-cluster ?. Or for spark standalone too ? El mar., 13 de mar. de 2018 18:34, Ruslan Dautkhanov escribió: > > Zeppelin version: 0.8.0 (merged at September 2017 version) > > https://issues.apache.org/jira/browse/ZEPPELIN-2898 was

Re: Zeppelin - Spark Driver location

> Zeppelin version: 0.8.0 (merged at September 2017 version) https://issues.apache.org/jira/browse/ZEPPELIN-2898 was merged end of September so not sure if you have that. Check out https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235 how to set this up. -- Ruslan Dautkhanov

Zeppelin - Spark Driver location

Hi zeppelin users ! I am working with zeppelin pointing to a spark in standalone. I am trying to figure out a way to make zeppelin runs the spark driver outside of client process that submits the application. According with the documentation (