Running Spark On Yarn without Spark-Submit

Hi, My requirement is to run Spark on Yarn without using the script spark-submit. I have a servlet and a tomcat server. As and when request comes, it creates a new SC and keeps it alive for the further requests, I ma setting my master in sparkConf as sparkConf.setMaster(yarn-cluster) but the

Re: Running Spark On Yarn without Spark-Submit

including u...@spark.apache.org. On Fri, Aug 29, 2014 at 2:03 PM, Archit Thakur archit279tha...@gmail.com wrote: Hi, My requirement is to run Spark on Yarn without using the script spark-submit. I have a servlet and a tomcat server. As and when request comes, it creates a new SC and

Re: Running Spark On Yarn without Spark-Submit

Archit We are using yarn-cluster mode , and calling spark via Client class directly from servlet server. It works fine. To establish a communication channel to give further requests, It should be possible with yarn client, but not with yarn server. Yarn client mode, spark driver