So here's a followup question : What's the preferred mode? We have a new cluster coming up with petabytes of data and we intend to take Spark to production. We are trying to figure out what mode would be safe and stable for production like environment. pros and cons? anyone?
Any reasons why one would chose Standalone over YARN? Thanks, Vipul On May 4, 2014, at 5:56 PM, Liu, Raymond <raymond....@intel.com> wrote: > In the core, they are not quite different > In standalone mode, you have spark master and spark worker who allocate > driver and executors for your spark app. > While in Yarn mode, Yarn resource manager and node manager do this work. > When the driver and executors have been launched, the rest part of resource > scheduling go through the same process, say between driver and executor > through akka actor. > > Best Regards, > Raymond Liu > > > -----Original Message----- > From: Sophia [mailto:sln-1...@163.com] > > Hey you guys, > What is the different in spark on yarn mode and standalone mode about > resource schedule? > Wish you happy everyday. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/different-in-spark-on-yarn-mode-and-standalone-mode-tp5300.html > Sent from the Apache Spark User List mailing list archive at Nabble.com.
