Hi, I was running spark application as a query service (much like
spark-shell but within my servlet container provided by spring-boot) with
spark 1.0.2 and standalone mode. Now After upgrading to spark 1.3.1 and
trying to use Yarn instead of standalone cluster things going south for me.
I created uber jar with all dependencies (spark-core, spark-yarn,
spring-boot) and tried to deploy my application.
15/07/29 11:19:26 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
15/07/29 11:19:27 INFO ipc.Client: Retrying connect to server:
0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
MILLISECONDS)
15/07/29 11:19:28 INFO ipc.Client: Retrying connect to server:
0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
MILLISECONDS)
15/07/29 11:19:29 INFO ipc.Client: Retrying connect to server:
0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
MILLISECONDS)
I also tried to exclude spark-yarn dependencies and supplied it during
runtime but same exception. We use MapR distribution and they said it's not
possible to run spark jobs on yarn without using spark-submit script. I can
try to launch my webapp using that script as my build artifact is
spring-boot jar (not war) but that just doesn't feel right. I should be
able to init service from my container not other way around.
--
[image: What's New with Xactly] <http://www.xactlycorp.com/email-click/>
<https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn]
<https://www.linkedin.com/company/xactly-corporation> [image: Twitter]
<https://twitter.com/Xactly> [image: Facebook]
<https://www.facebook.com/XactlyCorp> [image: YouTube]
<http://www.youtube.com/xactlycorporation>
