Are you running something on port 0 already? No actually I tired multiple ways to avoid this problem, and it seems to disappear when I m setting the num-executors to 6 (My hadoop cluster is 3 nodes), Could the num-executors have anything to do with the error I m getting??
On Sun, Jul 19, 2015 at 3:31 PM, ayan guha <guha.a...@gmail.com> wrote: > Are you running something on port 0 already? Looks like spark app master > is failing. > On 19 Jul 2015 06:13, "Chester @work" <ches...@alpinenow.com> wrote: > >> it might be a network issue. The error states failed to bind the server >> IP address >> >> Chester >> Sent from my iPhone >> >> On Jul 18, 2015, at 11:46 AM, Amjad ALSHABANI <ashshab...@gmail.com> >> wrote: >> >> Does anybody have any idea about the error I m having.. I am really >> clueless... And appreciate any idea :) >> >> Thanks in advance >> >> Amjad >> On Jul 17, 2015 5:37 PM, "Amjad ALSHABANI" <ashshab...@gmail.com> wrote: >> >>> Hello, >>> >>> First of all I m a newbie in Spark , >>> >>> I m trying to start the spark-shell with yarn cluster by running: >>> >>> $ spark-shell --master yarn-client >>> >>> Sometimes it goes well, but most of the time I got an error: >>> >>> Container exited with a non-zero exit code 10 >>> Failing this attempt. Failing the application. >>> ApplicationMaster host: N/A >>> ApplicationMaster RPC port: -1 >>> queue: default >>> start time: 1437145851944 >>> final status: FAILED >>> tracking URL: >>> http://My-HadoopServer:50080/cluster/app/application_1437033338028_0030 >>> user: hadoop >>> org.apache.spark.SparkException: Yarn application has already ended! It >>> might have been killed or unable to launch application master. >>> at >>> org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:115) >>> .... >>> .... >>> .... >>> >>> searching in the yarn logs I got this log >>> >>> $ yarn logs -applicationId application_1437033338028_0030 >>> 2015-07-17 17:11:03,961 - INFO >>> [sparkYarnAM-akka.actor.default-dispatcher-4:Slf4jLogger$$anonfun$receive$1$$anonfun$applyOrElse$3@74] >>> - Starting remoting >>> 2015-07-17 17:11:04,200 - ERROR >>> [sparkYarnAM-akka.actor.default-dispatcher-4:Slf4jLogger$$anonfun$receive$1$$anonfun$applyOrElse$1@65] >>> - failed to bind to My-HadoopServer/10.98.105.11:0, shutting down Netty >>> transport >>> 2015-07-17 17:11:04,210 - WARN [main:Logging$class@71] - Service >>> 'sparkYarnAM' could not bind on port 0. Attempting port 1. >>> ... >>> ... >>> ... >>> 2015-07-17 17:11:05,123 - ERROR [main:Logging$class@96] - Uncaught >>> exception: >>> java.net.BindException: Failed to bind to: >>> My-HadoopServer/HadoopServerIP:0: Service 'sparkYarnAM' failed after 16 >>> retries! >>> at >>> org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272) >>> at >>> akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:393) >>> at >>> akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:389) >>> at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) >>> ... >>> .... >>> .... >>> >>> >>> >>> I m using Spark 1.3, Hadoop 2.6 , >>> >>> and in spark-env.sh it points to my hadoop configuration: >>> >>> export HADOOP_CONF_DIR=/usr/hdp/2.2.4.4-16/hadoop/conf >>> >>> >>> Is this probleme coming from spark configuration or yarn configuration >>> (or spark with yarn confs)???? >>> >>> Any Ideas?? >>> >>> >>> >>> Amjad >>> >>>
