Spark has a first-class scheduler for Mesos, whereas it doesn't for Kubernetes. Running Spark on Kubernetes means running Spark in standalone mode, wrapped in a Kubernetes service: https://github.com/kubernetes/kubernetes/tree/master/examples/spark
So you're effectively comparing standalone vs. Mesos. For basic purposes, standalone works fine. Mesos adds support for things like docker images, security, resource reservations via roles, targeting specific nodes via attributes, etc. The main benefit of Mesos, however, is that you can share the same infrastructure with other, non-Spark services. We have users, for example, running Spark on the same cluster as HDFS, Cassandra, Kafka, web apps, Jenkins, etc. You can do this with Kubernetes to some extent, but running in standalone means that the Spark "partition" isn't elastic. You must statically partition to exclusively run Spark. On Sat, Aug 13, 2016 at 11:24 AM, guyoh <g12...@gmail.com> wrote: > My company is trying to decide whether to use kubernetes or mesos. Since we > are planning to use Spark in the near future, I was wandering what is the > best choice for us. > Thanks, > Guy > > > > -- > View this message in context: http://apache-spark-user-list. > 1001560.n3.nabble.com/mesos-or-kubernetes-tp27530.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- Michael Gummelt Software Engineer Mesosphere
