Hi JG, One way this can occur is that YARN doesn't have enough resources to run your job. Have you verified that it does? Are you able to submit using the same command from a node on the cluster?
-Sandy On Wed, Jul 8, 2015 at 3:19 PM, jegordon <jgordo...@gmail.com> wrote: > I'm trying to submit a spark job from a different server outside of my > Spark > Cluster (running spark 1.4.0, hadoop 2.4.0 and YARN) using the spark-submit > script : > > spark/bin/spark-submit --master yarn-client --executor-memory 4G > myjobScript.py > > The think is that my application never pass from the accepted state, it > stuck on it : > > 15/07/08 16:49:40 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:41 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:42 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:43 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:44 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:45 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:46 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:47 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:48 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > 15/07/08 16:49:49 INFO Client: Application report for > application_1436314873375_0030 (state: ACCEPTED) > > But if i execute the same script with spark-submit in the master server of > my cluster it runs correctly. > > I already set the yarn configuration in the remote server in > $YARN_CONF_DIR/yarn-site.xml like this : > > <property> > <name>yarn.resourcemanager.hostname</name> > <value>54.54.54.54</value> > </property> > > <property> > <name>yarn.resourcemanager.address</name> > <value>54.54.54.54:8032</value> > <description>Enter your ResourceManager hostname.</description> > </property> > > <property> > <name>yarn.resourcemanager.scheduler.address</name> > <value>54.54.54.54:8030</value> > <description>Enter your ResourceManager hostname.</description> > </property> > > <property> > <name>yarn.resourcemanager.resourcetracker.address</name> > <value>54.54.54.54:8031</value> > <description>Enter your ResourceManager hostname.</description> > </property> > Where 54.54.54.54 is the IP of my resourcemanager node. > > Why is this happening? do i have to configure something else in YARN to > accept remote submits? or what am i missing? > > Thanks a lot > > JG > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Remote-spark-submit-not-working-with-YARN-tp23728.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
