Thanks for the discussion and the feedback, everyone! I sense a general consensus, so am going to go ahead with the branch proposal.
Please regard this as the conclusion of the discussion here. If you have more inputs, feel free to jump on to the JIRA YARN-5079! Thanks again, +Vinod > On Jul 14, 2016, at 7:36 PM, Vinod Kumar Vavilapalli <vino...@apache.org> > wrote: > > Hi, Hadoop YARN community! > > (Cross-posting across Hadoop and Slider communities) > > I opened a JIRA a little while ago to pursue a native YARN framework for > services: https://issues.apache.org/jira/browse/YARN-5079. > > It is part of a bigger effort that a bunch of us YARN community members are > interested in making progress on: YARN-4692 - [Umbrella] Simplified and > first-class support for services in YARN. > > The idea is that with our current attention on making services first-class, > it's time to take a fresh look at how we can make Apache Hadoop YARN support > services well out of the box. I’ve been looking at various possibilities - > ranging from a custom new framework room scratch to using one of the existing > projects - and stopped at Apache Slider (http://slider.incubator.apache.org) > given its association with some of the YARN community members (Steve > Loughran, Devaraj Das, Arun C Murthy, myself etc.). > > Slider client & AM already handles a great deal of the functionality that we > need. I posit that assimilating the client, ApplicationMaster etc of an > existing framework like Apache Slider can serve our purpose really well. My > early informal discussions about this with few Hadoop and Slider community > members yielded generally favourable feedback. > > The Apache Slider incubator community also discussed this and expressed > generally positive interest in YARN taking up Slider’s key pieces, you can > see that discussion here: https://s.apache.org/0hoh. > > So in summary, we are looking to the following > > - Code > — ‘Graduate' key pieces (Slider client, AM) of Apache Slider into Apache > Hadoop for providing a native services experiences in YARN > — Leave for now some of the pieces behind in Apache Slider - (a) Slider > agent as we won’t need it, (b) Slider packages that need more deliberation in > terms of where they will live in the long term. > — Create a branch in YARN, copy this code over into a new module(s), and > work towards completing a functioning app running on YARN. > > - Communities & releases > — Good thing is that many of Apache Slider community members are already > seasoned folk in the Apache Hadoop ecosystem projects. For those committers & > PMC in Slider that are not yet Hadoop committers / PMC, without complicating > things much, a proposed path forward is active participation in the branch > (as branch committers?) and eventually in mainline YARN and thus go through a > natural progression to committership / PMC. Given that most of the members > are stalwarts in the Apache communities, this should be a cinch IMO. > — The work on this new code can start, and depending on its state, and > assuming that the experiment succeeds, can be merged into trunk and later > picked up in the next nearest & feasible Apache Hadoop release. > — While the work on forked-over-code goes on till we have a functioning > app, the current Apache Slider project continues to live, with supporting > releases etc. At some point in the future, when things become clearer, more > decisions can be taken on the (parts left behind in the) incubating project’s > future. > > Thoughts? > > Thanks > +Vinod --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
