Hi Nik, Bharath is mostly referring to Spark commiters in this thread.
Tim On Tue, Jun 9, 2015 at 9:51 PM, Niklas Nielsen <nik...@mesosphere.io> wrote: > Hi Bharath (and rest of Spark dev list!), > > Just a small shout out: I am a Apache Mesos Committer and would love to help > out with anything you need to get this going. > > Cheers, > Nik > > On 9 June 2015 at 21:10, Bharath Ravi Kumar <reachb...@gmail.com> wrote: >> >> All, >> >> Despite the common origin of spark & mesos, the stability and adoption of >> mesos, and the age of the spark-mesos binding, I find the mesos support less >> mature, with fundamental shortcomings (like framework auth) remaining >> unresolved. If there's shortage of developer time, I'd be glad to >> contribute, but it's unclear if the committer group has sufficient time (and >> priority) to take the mesos support forward. While it has been stated often >> that support for mesos & yarn are equally important, that doesn't seem to >> translate to visible progress. I'd be glad if my observation is incorrect as >> I seek better focus and long term commitment on the mesos support. >> As for the specific issue (6284), I'm happy to build, testing & eventually >> deploy the patch in our production cluster, but I'd rather see it becoming >> mainstream. >> Thanks for your consideration. >> >> -Bharath >> >> >> On Thu, May 28, 2015 at 9:18 AM, Bharath Ravi Kumar <reachb...@gmail.com> >> wrote: >>> >>> A follow up : considering that spark on mesos is indeed important to >>> databricks, its partners and the community, fundamental issues like >>> spark-6284 shouldn't be languishing for this long. A mesos cluster hosting >>> diverse (i.e.multi-tenant) workloads is a common scenario in production for >>> serious users. The ability to auth a framework & assign roles would be a >>> fairly basic ask, one would imagine. Is the lack of time / effort a >>> constraint? If so, I'd be glad to help (as mentioned in the jira). >>> >>> On Fri, May 15, 2015 at 5:29 PM, Iulian Dragoș >>> <iulian.dra...@typesafe.com> wrote: >>>> >>>> Hi Ankur, >>>> >>>> Just to add a thought to Tim's excellent answer, Spark on Mesos is very >>>> important to us and is the recommended deployment for our customers as >>>> Typesafe. >>>> >>>> Thanks for pointing to your PR, I see Tim already went through a round >>>> of reviews. It seems very useful, I'll give it a try as well. >>>> >>>> thanks, >>>> iulian >>>> >>>> >>>> >>>> On Fri, May 15, 2015 at 9:53 AM, Ankur Chauhan <an...@malloc64.com> >>>> wrote: >>>>> >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA1 >>>>> >>>>> Hi Tim, >>>>> >>>>> Thanks for such a detailed email. I am excited to hear about the new >>>>> features, I had a pull request going for adding "attribute based >>>>> filtering in the mesos scheduler" but it hasn't received much love - >>>>> https://github.com/apache/spark/pull/5563 . I am a fan of >>>>> mesos/marathon/mesosphere and spark ecosystems and trying to push >>>>> adoption at my workplace. >>>>> >>>>> It would love to see documentation, tutorials (anything actually) that >>>>> would make mesos + spark a better and more fleshed out solution. Would >>>>> it be possible for you to share some links to the JIRA and pull >>>>> requests so that I can keep track on the progress/features. >>>>> >>>>> Again, thanks for replying. >>>>> >>>>> - -- Ankur Chauhan >>>>> >>>>> On 15/05/2015 00:39, Tim Chen wrote: >>>>> > Hi Ankur, >>>>> > >>>>> > This is a great question as I've heard similar concerns about Spark >>>>> > on Mesos. >>>>> > >>>>> > At the time when I started to contribute to Spark on Mesos approx >>>>> > half year ago, the Mesos scheduler and related code hasn't really >>>>> > got much attention from anyone and it was pretty much in >>>>> > maintenance mode. >>>>> > >>>>> > As a Mesos PMC that is really interested in Spark I started to >>>>> > refactor and check out different JIRAs and PRs around the Mesos >>>>> > scheduler, and after that started to fix various bugs in Spark, >>>>> > added documentation and also in fix related Mesos issues as well. >>>>> > >>>>> > Just recently for 1.4 we've merged in Cluster mode and Docker >>>>> > support, and there are also pending PRs around framework >>>>> > authentication, multi-role support, dynamic allocation, more finer >>>>> > tuned coarse grain mode scheduling configurations, etc. >>>>> > >>>>> > And finally just want to mention that Mesosphere and Typesafe is >>>>> > collaborating to bring a certified distribution >>>>> > >>>>> > (https://databricks.com/spark/certification/certified-spark-distributi >>>>> on) >>>>> > of Spark on Mesos and DCOS, and we will be pouring resources into >>>>> > not just maintain Spark on Mesos but drive more features into the >>>>> > Mesos scheduler and also in Mesos so stateful services can leverage >>>>> > new APIs and features to make better scheduling decisions and >>>>> > optimizations. >>>>> > >>>>> > I don't have a solidified roadmap to share yet, but we will be >>>>> > discussing this and hopefully can share with the community soon. >>>>> > >>>>> > In summary Spark on Mesos is not dead or in maintenance mode, and >>>>> > look forward to see a lot more changes from us and the community. >>>>> > >>>>> > Tim >>>>> > >>>>> > On Thu, May 14, 2015 at 11:30 PM, Ankur Chauhan >>>>> > <an...@malloc64.com <mailto:an...@malloc64.com>> wrote: >>>>> > >>>>> > Hi, >>>>> > >>>>> > This is both a survey type as well as a roadmap query question. It >>>>> > seems like of the cluster options to run spark (i.e. via YARN and >>>>> > Mesos), YARN seems to be getting a lot more attention and patches >>>>> > when compared to Mesos. >>>>> > >>>>> > Would it be correct to assume that spark on mesos is more or less >>>>> > a dead or something like a maintenance-only feature and YARN is >>>>> > the recommended way to go? >>>>> > >>>>> > What is the roadmap for spark on mesos? and what is the roadmap >>>>> > for spark on yarn. I like mesos so as much as I would like to see >>>>> > it thrive I don't think spark community is active (or maybe it >>>>> > just appears that way). >>>>> > >>>>> > Another more community oriented question: what do most people use >>>>> > to run spark in production or more-than-POC products? Why did you >>>>> > make that decision? >>>>> > >>>>> > There was a similar post form early 2014 where Metei answered that >>>>> > mesos and yarn were equally important, but has this changed as >>>>> > spark has now reached almost 1.4.0 stage? >>>>> > >>>>> > -- Ankur Chauhan >>>>> > >>>>> > --------------------------------------------------------------------- >>>>> > >>>>> > >>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>> > <mailto:user-unsubscr...@spark.apache.org> For additional commands, >>>>> > e-mail: user-h...@spark.apache.org >>>>> > <mailto:user-h...@spark.apache.org> >>>>> > >>>>> > >>>>> -----BEGIN PGP SIGNATURE----- >>>>> >>>>> iQEcBAEBAgAGBQJVVaXvAAoJEOSJAMhvLp3LzkIH/RLPVUoOcFo0Gij1NpZDszN4 >>>>> xWvesbOZszuqD8H1Dhyndz4RQKnrodyCE+NycFB+utd9epmuyGemmHpTnq18Gek6 >>>>> PR5jqmgza94dOy0rfuIVvba14ALZb4tO9SgkjyGujrpMlFYvxTjBYdYCAjfEOTx7 >>>>> A/vqaCzPSBRBmO8gWx07GWa4zI70qBSZ9KnV7dgtqfUUgPKdF4NnMZWRJjTO9Bp8 >>>>> tTmWMldqYPqI95wdeeqTGMH0XT6JAKAiCskf62DGadRBsOshrhmh5mAQzUFwoTpA >>>>> w4uZ+qMrTsblBvOf9z++v0eY8VBiQpOyXfOBiYiCNRtSsGa0KvqwgF1S/yLeRs0= >>>>> =4Aax >>>>> -----END PGP SIGNATURE----- >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> -- >>>> Iulian Dragos >>>> >>>> ------ >>>> Reactive Apps on the JVM >>>> www.typesafe.com >>>> >>> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
