ZhuGe, If you run your program in the "cluster" deploy-mode you get resiliency against driver failure, though there are some steps you have to take in how you write your streaming job to allow for transparent resume. Netflix did a nice writeup of this resiliency here<http://techblog.netflix.com/2015/03/can-spark-streaming-survive-chaos-monkey.html>.
If you tie in ZooKeeper you can also get resiliency against Master failure, which has some documentation here<http://spark.apache.org/docs/1.4.0/spark-standalone.html>. Regards, -- Matthew ________________________________ From: ZhuGe [[email protected]] Sent: Tuesday, July 21, 2015 3:07 AM To: [email protected] Subject: Would driver shutdown cause app dead? Hi all: I am a bit confuse about the work of driver. In our productin enviroment, we have a spark streaming app running in standone mode. what we concern is that if the driver shutdown accidently(host shutdown or whatever). would the app running normally? Any explanation would be appreciated!! Cheers --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
