Dear Akhil Das-2, My application runs in standalone mode, with 50 machines. It's okay if the input file is small, but if i increases the input to 8GB, the application just serveral iterations, and then print following error logs:
14/10/20 17:15:28 WARN AppClient$ClientActor: Connection to akka.tcp://sparkMaster@master1:7077 failed; waiting for master to reconnect... 14/10/20 17:15:28 WARN SparkDeploySchedulerBackend: Disconnected from Spark cluster! Waiting for reconnection... 14/10/20 17:15:28 WARN AppClient$ClientActor: Connection to akka.tcp://sparkMaster@master1:7077 failed; waiting for master to reconnect... 14/10/20 17:15:30 INFO SparkDeploySchedulerBackend: Executor 10 disconnected, so removing it 14/10/20 17:15:30 ERROR TaskSchedulerImpl: Lost executor 10 on slave46: remote Akka client disassociated 14/10/20 17:15:30 INFO TaskSetManager: Re-queueing tasks for 10 from TaskSet 533.0 14/10/20 17:15:30 WARN TaskSetManager: Lost TID 4475 (task 533.0:20) 14/10/20 17:15:30 INFO DAGScheduler: Executor lost: 10 (epoch 80) 14/10/20 17:15:30 INFO BlockManagerMasterActor: Trying to remove executor 10 from BlockManagerMaster. 14/10/20 17:15:30 INFO BlockManagerMaster: Removed 10 successfully in removeExecutor 14/10/20 17:15:30 INFO Stage: Stage 563 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 590 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 585 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 580 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 541 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 581 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 576 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 571 is now unavailable on executor 10 (44/45, false) 14/10/20 17:15:30 INFO Stage: Stage 554 is now unavailable on executor 10 (44/45, false) Akhil Das-2 wrote > What is the application that you are running? and what is the cluster > setup > that you are having? Given the logs, it looks like the master is dead for > some reason. > > Thanks > Best Regards > > On Sun, Oct 19, 2014 at 2:48 PM, randylu < > randylu26@ > > wrote: > >> In additional, driver receives serveral DisassociatedEvent messages. >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/why-does-driver-connects-to-master-fail-tp16758p16759.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: > user-unsubscribe@.apache >> For additional commands, e-mail: > user-help@.apache >> >> -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/why-does-driver-connects-to-master-fail-tp16758p16802.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
