Hi Jai,
Refer this doc and make sure your network is not blocking
http://apache-spark-user-list.1001560.n3.nabble.com/Submitting-Spark-job-on-Unix-cluster-from-dev-environment-Windows-td16989.html
Also make sure you are using the same version of spark in both places (the
one on the cluster, and the one that you used inside your application)
Thanks
Best Regards
On Tue, Dec 16, 2014 at 1:25 PM, Jai <jaidishhari...@gmail.com> wrote:
>
> Hi
>
> I have installed a standalone Spark set up in standalone mode in a Linux
> server and I am trying to access that spark setup from Java in windows.
> When
> I try connecting to Spark I see the following exception
>
> 14/12/16 12:52:52 WARN TaskSchedulerImpl: Initial job has not accepted any
> resources; check your cluster UI to ensure that workers are registered and
> have sufficient memory
> 14/12/16 12:52:56 INFO AppClient$ClientActor: Connecting to master
> spark://01hw294954.INDIA:7077...
> 14/12/16 12:53:07 WARN TaskSchedulerImpl: Initial job has not accepted any
> resources; check your cluster UI to ensure that workers are registered and
> have sufficient memory
> 14/12/16 12:53:16 INFO AppClient$ClientActor: Connecting to master
> spark://01hw294954.INDIA:7077...
> 14/12/16 12:53:22 WARN TaskSchedulerImpl: Initial job has not accepted any
> resources; check your cluster UI to ensure that workers are registered and
> have sufficient memory
> 14/12/16 12:53:36 ERROR SparkDeploySchedulerBackend: Application has been
> killed. Reason: All masters are unresponsive! Giving up.
> 14/12/16 12:53:36 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks
> have all completed, from pool
> 14/12/16 12:53:36 INFO TaskSchedulerImpl: Cancelling stage 0
> 14/12/16 12:53:36 INFO DAGScheduler: Failed to run collect at
> MySqlConnector.java:579
> Exception in thread "main" org.apache.spark.SparkException: Job aborted due
> to stage failure: All masters are unresponsive! Giving up.
> at
> org.apache.spark.scheduler.DAGScheduler.org
> $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1033)
> at
>
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1017)
> at
>
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1015)
> at
>
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at
> scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1015)
> at
>
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633)
> at
>
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:633)
> at scala.Option.foreach(Option.scala:236)
> at
>
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:633)
> at
>
> org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1207)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
> at akka.actor.ActorCell.invoke(ActorCell.scala:456)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
> at akka.dispatch.Mailbox.run(Mailbox.scala:219)
> at
>
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
> at
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
>
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at scala.concurrent.forkjoin.ForkJoinWorkerThread.run
>
> I have attached the Spark Master UI
>
> Spark Master at spark://01hw294954.INDIA:7077
> URL: spark://01hw294954.INDIA:7077
> Workers: 1
> Cores: 2 Total, 0 Used
> Memory: 835.0 MB Total, 0.0 B Used
> Applications: 0 Running, 0 Completed
> Drivers: 0 Running, 0 Completed
> Status: ALIVE
> Workers
>
> Id Address State Cores Memory
> worker-20141216123503-01hw294954.INDIA-38962 01hw294954.INDIA:38962
> ALIVE 2
> (0 Used) 835.0 MB (0.0 B Used)
> Running Applications
>
> ID Name Cores Memory per Node Submitted Time User State
> Duration
> Completed Applications
>
> ID Name Cores Memory per Node Submitted Time User State
> Duration
>
>
> My Spark Slave is
>
> Spark Worker at 01hw294954.INDIA:38962
> ID: worker-20141216123503-01hw294954.INDIA-38962
> Master URL: spark://01hw294954.INDIA:7077
> Cores: 2 (0 Used)
> Memory: 835.0 MB (0.0 B Used)
> Back to Master
>
> Running Executors (0)
>
> ExecutorID Cores State Memory Job Details Logs
>
>
> My Java Master Code looks like this
>
> SparkConf sparkConf = new SparkConf().setAppName("JdbcRddTest");
> sparkConf.setMaster("spark://01hw294954.INDIA:7077");
> When I tried using the same code with the local spark set up as the master
> it ran.
>
> Any help for solving this issue is very much appreciated.
>
> Thanks and Regards
> Jai
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Accessing-Apache-Spark-from-Java-tp20700.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
