The standalone mode is against Yarn mode or Mesos mode, which means spark uses Yarn or Mesos as cluster managements.
Local mode is actually a standalone mode which everything runs on the single local machine instead of remote clusters. That is my understanding. On Sat, Jun 11, 2016 at 12:40 PM, Ashok Kumar <ashok34...@yahoo.com.invalid> wrote: > Thank you for grateful > > I know I can start spark-shell by launching the shell itself > > spark-shell > > Now I know that in standalone mode I can also connect to master > > spark-shell --master spark://<HOST>:7077 > > My point is what are the differences between these two start-up modes for > spark-shell? If I start spark-shell and connect to master what performance > gain will I get if any or it does not matter. Is it the same as for > spark-submit > > > > regards > > > On Saturday, 11 June 2016, 19:39, Mohammad Tariq <donta...@gmail.com> > wrote: > > > Hi Ashok, > > In local mode all the processes run inside a single jvm, whereas in > standalone mode we have separate master and worker processes running in > their own jvms. > > To quickly test your code from within your IDE you could probable use the > local mode. However, to get a real feel of how Spark operates I would > suggest you to have a standalone setup as well. It's just the matter > of launching a standalone cluster either manually(by starting a master and > workers by hand), or by using the launch scripts provided with Spark > package. > > You can find more on this *here* > <http://spark.apache.org/docs/latest/spark-standalone.html>. > > HTH > > > > [image: http://] > > Tariq, Mohammad > about.me/mti > [image: http://] > <http://about.me/mti> > > > On Sat, Jun 11, 2016 at 11:38 PM, Ashok Kumar < > ashok34...@yahoo.com.invalid> wrote: > > Hi, > > What is the difference between running Spark in Local mode or standalone > mode? > > Are they the same. If they are not which is best suited for non prod work. > > I am also aware that one can run Spark in Yarn mode as well. > > Thanks > > > > >
