Thanks Ted. Adding dependency to spark-network-yarn would allow resolution
to YarnShuffleService which from docs suggests that it runs on Node Manager
and perhaps isn't useful while submitting jobs programmatically. I think
what I need is a dependency to spark-yarn module so that classes like
YarnRMClient, Client, etc resolve. These are used while submitting jobs to
YARN atleast in yarn-client mode. Any particular reason why spark-yarn
client module isn't published anymore?
On Mon Dec 29 2014 at 13:00:26 Ted Yu wrote:
> See this thread:
> http://search-hadoop.com/m/JW1q5vd61V1/Spark-yarn+1.2.0&subj=Re+spark+yarn_2+10+1+2+0+artifacts
>
> Cheers
>
> On Dec 28, 2014, at 11:13 PM, Aniket Bhatnagar
> wrote:
>
> Hi all
>
> I just realized that spark-yarn artifact hasn't been published for 1.2.0
> release. Any particular reason for that? I was using it in my yet another
> spark-job-server project to submit jobs to a YARN cluster through
> convenient REST APIs (with some success). The job server was creating
> SparkContext with master as 'yarn-client' and hence needed to be dependent
> on spark-yarn jar.
>
> Thanks,
> Aniket
>
>
>
