Hi Jonathan, Thanks for this information! I will take a look into it. However is there a way to reconnect the lost node? Or there's no way that I could do to find back the lost worker?
Thanks! Anny On Wed, Jun 24, 2015 at 6:06 PM, Kelly, Jonathan <jonat...@amazon.com> wrote: > Just curious, would you be able to use Spark on EMR rather than on EC2? > Spark on EMR will handle lost nodes for you, and it will let you scale > your cluster up and down or clone a cluster (its config, that is, not the > data stored in HDFS), among other things. We also recently announced > official support for Spark on EMR: http://aws.amazon.com/emr/spark > > ~ Jonathan Kelly (from Amazon AWS EMR) > > > On 6/24/15, 5:58 PM, "anny9699" <anny9...@gmail.com> wrote: > > >Hi, > > > >According to the Spark UI, one worker is lost after a failed job. It is > >not > >a "lost executor" error, but that the UI now only shows 8 workers (I have > >9 > >workers). However from the ec2 console, it shows the machine is "running" > >and no check alarms. So I am confused how I could reconnect the lost > >machine > >in aws ec2? > > > >I met this problem before, and my solution was to rebuilt a new cluster. > >However now it is a little hard to rebuild a cluster, so I am wondering if > >there's some way to find back the lost machine? > > > >Thanks a lot! > > > > > > > >-- > >View this message in context: > > > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-ec2-cluster-lost > >-worker-tp23482.html > >Sent from the Apache Spark User List mailing list archive at Nabble.com. > > > >--------------------------------------------------------------------- > >To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > >For additional commands, e-mail: user-h...@spark.apache.org > > > >
