I'd like to get in 2.0.3 the JIRAs that enable pluggable shuffle and pluggable sort: MAPREDUCE-4049, MAPREDUCE-4809, MAPREDUCE-4807 & MAPREDUCE-4808
Unless I hear otherwise, I'd be merging those commits in branch-2 tomorrow mid afternoon PST. Thx On Wed, Jan 16, 2013 at 10:38 AM, Arun C Murthy <[email protected]> wrote: > Seems reasonable since it's a small patch and has had one round of review. > > I spoke to Todd yesterday and he seemed to think it can be pushed in > fairly soon too. So that's two of you. > > Anyway, since I'm still awaiting HADOOP-9215 too, I'd appreciate if we can > get HDFS-4404 committed asap. > > thanks, > Arun > > On Jan 16, 2013, at 10:30 AM, Suresh Srinivas wrote: > > > Arun, > > > > HDFS-4404 is not marked as a blocker. It is a serious bug and I would > like > > to make it a blocker. Let me know if you are okay. > > > > If folks do not want to hold the release for that fix, lets at least try > to > > capture that in release notes and try get that fix into 2.0.4. > > > > Regards, > > Suresh > > > > > > On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <[email protected]> > wrote: > > > >> Happy new year folks! > >> > >> I'm glad to see that we are down to the last couple of blockers I hope > we > >> can resolve in the next 24-hrs or so. > >> > >> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for > >> commits targeted towards the next release, create the new fix-versions > in > >> jira and spin the RC. > >> Committers - after the branch is created, please use only the new > >> fix-version (2.0.4) and check with me before you commit to > >> branch-2.0.3-alpha. Thanks. > >> > >> As always, I'd appreciate help to resolve any unexpected surprises and > >> also to help verify the RC. > >> > >> Hopefully we can start the new year with a great release, there are lots > >> of goodies in 2.0.3-alpha. > >> > >> thanks, > >> Arun > >> > >> On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote: > >> > >>> As Sid responded I think we can move off alpha once we fix > >> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but > none > >> as egregious as those two. > >>> > >>> Someone on my team is starting on it as we speak and I believe we can > >> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By > then > >> we'll also have wider rollouts of YARN and would have fixed some more > >> issues we've seen at very high scale deployments at Y!. Sounds like the > >> right time to do a beta release to me. > >>> > >>> Arun > >>> > >>> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote: > >>> > >>>> Hey Arun, > >>>> > >>>> Awesome to see we're almost down to zero blockers. What are your > >> thoughts > >>>> on removing the "alpha" label from the upcoming release? It seems to > me > >>>> from the earlier discussion that most folks feel that we're at the > point > >>>> where the interfaces are sufficiently stable to warrant it. > >>>> > >>>> Aaron > >>>> > >>>> > >>>> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <[email protected]> > >> wrote: > >>>> > >>>>> Nearly there: > >>>>> http://s.apache.org/hadoop-2-blockers > >>>>> > >>>>> YARN-217 should be easy, I'd also like to get in YARN-253. > >>>>> > >>>>> Arun > >>>>> > >>>>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote: > >>>>> > >>>>>> Any news on how this is progressing? Some folks in this thread below > >>>>>> inquired about getting this release out around the New Year > timeframe, > >>>>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all > >>>>>> know how long lifecycle changes can take to get pushed through ;-) > >>>>>> > >>>>>> -Todd > >>>>>> > >>>>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran > >>>>>> <[email protected]> wrote: > >>>>>>> I want to make some changes to the lifecycle of a yarn service (in > a > >>>>>>> backwards compatible way). > >>>>>>> > >>>>>>> https://issues.apache.org/jira/browse/YARN-117 > >>>>>>> > >>>>>>> > >>>>>>> 1. formal state machine model with stop state idempotent and > >>>>> entry-able > >>>>>>> from any state > >>>>>>> 2. waiting/blocked state a service can enter when waiting for > >>>>> something > >>>>>>> else > >>>>>>> 3. an alternate base class that does the state model checks before > >>>>>>> executing any state change functions -currently its done at > >>>>>>> end-of-operation in the super() calls. > >>>>>>> 4. gradual move of services to the stricter base class. > >>>>>>> > >>>>>>> With a new base class nothing will break (as the move can be done > >>>>>>> case-by-case, leaving the heavily subclassed ones alone); the state > >>>>> model > >>>>>>> extensions & formalisation would be visible but not used. > >>>>>>> > >>>>>>> I don't want to hold anything up, because I need more testing of > >> things > >>>>>>> before this is ready for review. I just want to get the fixes in > >> before > >>>>> it > >>>>>>> ships > >>>>>>> > >>>>>>> On 19 November 2012 16:22, Robert Evans <[email protected]> > wrote: > >>>>>>> > >>>>>>>> I am OK with removing the alpha assuming that we think that the > APIs > >>>>> are > >>>>>>>> stable enough that we are willing to truly start maintaining > >> backwards > >>>>>>>> compatibility on them within 2.X. From what I have seen I think > that > >>>>> they > >>>>>>>> are fairly stable and I think there is enough adoption by other > >>>>> projects > >>>>>>>> right now that breaking backwards compatibility would be > >> problematic. > >>>>>>>> > >>>>>>>> --Bobby Evans > >>>>>>>> > >>>>>>>> On 11/16/12 11:34 PM, "Stack" <[email protected]> wrote: > >>>>>>>> > >>>>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers < > [email protected]> > >>>>> wrote: > >>>>>>>>>> Hi Arun, > >>>>>>>>>> > >>>>>>>>>> Given that the 2.0.3 release is intended to reflect the growing > >>>>>>>>>> stability > >>>>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which > >> provides a > >>>>>>>>>> complete HDFS HA solution, I think it's time we consider > removing > >> the > >>>>>>>>>> "-alpha" label from the release version. My preference would be > to > >>>>>>>>>> remove > >>>>>>>>>> the label entirely, but we could also perhaps call it "-beta" or > >>>>>>>>>> something. > >>>>>>>>>> > >>>>>>>>>> Thoughts? > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> I think it fine after two minor releases undoing the '-alpha' > >> suffix. > >>>>>>>>> > >>>>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all > >>>>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x. > >>>>>>>>> > >>>>>>>>> St.Ack > >>>>>>>> > >>>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Todd Lipcon > >>>>>> Software Engineer, Cloudera > >>>>> > >>>>> -- > >>>>> Arun C. Murthy > >>>>> Hortonworks Inc. > >>>>> http://hortonworks.com/ > >>>>> > >>>>> > >>>>> > >>> > >>> -- > >>> Arun C. Murthy > >>> Hortonworks Inc. > >>> http://hortonworks.com/ > >>> > >>> > >> > >> -- > >> Arun C. Murthy > >> Hortonworks Inc. > >> http://hortonworks.com/ > >> > >> > >> > > > > > > -- > > http://hortonworks.com/download/ > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > -- Alejandro
