Thanks Ted. Adding dependency to spark-network-yarn would allow resolution to YarnShuffleService which from docs suggests that it runs on Node Manager and perhaps isn't useful while submitting jobs programmatically. I think what I need is a dependency to spark-yarn module so that classes like YarnRMClient, Client, etc resolve. These are used while submitting jobs to YARN atleast in yarn-client mode. Any particular reason why spark-yarn client module isn't published anymore?
On Mon Dec 29 2014 at 13:00:26 Ted Yu <yuzhih...@gmail.com> wrote: > See this thread: > http://search-hadoop.com/m/JW1q5vd61V1/Spark-yarn+1.2.0&subj=Re+spark+yarn_2+10+1+2+0+artifacts > > Cheers > > On Dec 28, 2014, at 11:13 PM, Aniket Bhatnagar <aniket.bhatna...@gmail.com> > wrote: > > Hi all > > I just realized that spark-yarn artifact hasn't been published for 1.2.0 > release. Any particular reason for that? I was using it in my yet another > spark-job-server project to submit jobs to a YARN cluster through > convenient REST APIs (with some success). The job server was creating > SparkContext with master as 'yarn-client' and hence needed to be dependent > on spark-yarn jar. > > Thanks, > Aniket > > >
