-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Tim,
Thanks for such a detailed email. I am excited to hear about the new features, I had a pull request going for adding "attribute based filtering in the mesos scheduler" but it hasn't received much love - https://github.com/apache/spark/pull/5563 . I am a fan of mesos/marathon/mesosphere and spark ecosystems and trying to push adoption at my workplace. It would love to see documentation, tutorials (anything actually) that would make mesos + spark a better and more fleshed out solution. Would it be possible for you to share some links to the JIRA and pull requests so that I can keep track on the progress/features. Again, thanks for replying. - -- Ankur Chauhan On 15/05/2015 00:39, Tim Chen wrote: > Hi Ankur, > > This is a great question as I've heard similar concerns about Spark > on Mesos. > > At the time when I started to contribute to Spark on Mesos approx > half year ago, the Mesos scheduler and related code hasn't really > got much attention from anyone and it was pretty much in > maintenance mode. > > As a Mesos PMC that is really interested in Spark I started to > refactor and check out different JIRAs and PRs around the Mesos > scheduler, and after that started to fix various bugs in Spark, > added documentation and also in fix related Mesos issues as well. > > Just recently for 1.4 we've merged in Cluster mode and Docker > support, and there are also pending PRs around framework > authentication, multi-role support, dynamic allocation, more finer > tuned coarse grain mode scheduling configurations, etc. > > And finally just want to mention that Mesosphere and Typesafe is > collaborating to bring a certified distribution > (https://databricks.com/spark/certification/certified-spark-distributi on) > of Spark on Mesos and DCOS, and we will be pouring resources into > not just maintain Spark on Mesos but drive more features into the > Mesos scheduler and also in Mesos so stateful services can leverage > new APIs and features to make better scheduling decisions and > optimizations. > > I don't have a solidified roadmap to share yet, but we will be > discussing this and hopefully can share with the community soon. > > In summary Spark on Mesos is not dead or in maintenance mode, and > look forward to see a lot more changes from us and the community. > > Tim > > On Thu, May 14, 2015 at 11:30 PM, Ankur Chauhan > <an...@malloc64.com <mailto:an...@malloc64.com>> wrote: > > Hi, > > This is both a survey type as well as a roadmap query question. It > seems like of the cluster options to run spark (i.e. via YARN and > Mesos), YARN seems to be getting a lot more attention and patches > when compared to Mesos. > > Would it be correct to assume that spark on mesos is more or less > a dead or something like a maintenance-only feature and YARN is > the recommended way to go? > > What is the roadmap for spark on mesos? and what is the roadmap > for spark on yarn. I like mesos so as much as I would like to see > it thrive I don't think spark community is active (or maybe it > just appears that way). > > Another more community oriented question: what do most people use > to run spark in production or more-than-POC products? Why did you > make that decision? > > There was a similar post form early 2014 where Metei answered that > mesos and yarn were equally important, but has this changed as > spark has now reached almost 1.4.0 stage? > > -- Ankur Chauhan > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > <mailto:user-unsubscr...@spark.apache.org> For additional commands, > e-mail: user-h...@spark.apache.org > <mailto:user-h...@spark.apache.org> > > -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJVVaXvAAoJEOSJAMhvLp3LzkIH/RLPVUoOcFo0Gij1NpZDszN4 xWvesbOZszuqD8H1Dhyndz4RQKnrodyCE+NycFB+utd9epmuyGemmHpTnq18Gek6 PR5jqmgza94dOy0rfuIVvba14ALZb4tO9SgkjyGujrpMlFYvxTjBYdYCAjfEOTx7 A/vqaCzPSBRBmO8gWx07GWa4zI70qBSZ9KnV7dgtqfUUgPKdF4NnMZWRJjTO9Bp8 tTmWMldqYPqI95wdeeqTGMH0XT6JAKAiCskf62DGadRBsOshrhmh5mAQzUFwoTpA w4uZ+qMrTsblBvOf9z++v0eY8VBiQpOyXfOBiYiCNRtSsGa0KvqwgF1S/yLeRs0= =4Aax -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
