Hi All We are down to 3 Blockers and 4 Critical now. Thanks all of you for helping in this. I am following up on these tickets, once its closed we will cut the 3.2 branch.
Thanks Sunil Govindan On Wed, Sep 12, 2018 at 5:10 PM Sunil G <sun...@apache.org> wrote: > Hi All, > > Inline with the original 3.2 communication proposal dated 17th July 2018, > I would like to provide more updates. > > We are approaching previously proposed code freeze date (September 14, > 2018). So I would like to cut 3.2 branch on 17th Sept and point existing > trunk to 3.3 if there are no issues. > > *Current Release Plan:* > Feature freeze date : all features to merge by September 7, 2018. > Code freeze date : blockers/critical only, no improvements and > blocker/critical bug-fixes September 14, 2018. > Release date: September 28, 2018 > > If any critical/blocker tickets which are targeted to 3.2.0, we need to > backport to 3.2 post branch cut. > > Here's an updated 3.2.0 feature status: > > 1. Merged & Completed features: > > - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning workloads > Initial cut. > - (Uma) HDFS-10285: HDFS Storage Policy Satisfier > - (Sunil) YARN-7494: Multi Node scheduling support in Capacity Scheduler. > - (Chandni/Eric) YARN-7512: Support service upgrade via YARN Service API > and CLI. > - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. > - (Inigo) HDFS-12615: Router-based HDFS federation. Improvement works. > > 2. Features close to finish: > > - (Steve) S3Guard Phase III. Close to commit. > - (Steve) S3a phase V. Close to commit. > - (Steve) Support Windows Azure Storage. Close to commit. > > 3. Tentative/Cancelled features for 3.2: > - (Rohith) YARN-5742: Serve aggregated logs of historical apps from ATSv2. > Patch in progress. > - (Haibo Chen) YARN-1011: Resource overcommitment. Looks challenging to be > done before Aug 2018. > - (Eric) YARN-7129: Application Catalog for YARN applications. Challenging > as more discussions are on-going. > > *Summary of 3.2.0 issues status:* > 19 Blocker and Critical issues [1] are open, I am following up with owners > to get status on each of them to get in by Code Freeze date. > > [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker, > Critical) AND resolution = Unresolved AND "Target Version/s" = 3.2.0 ORDER > BY priority DESC > > Thanks, > Sunil > > > > On Thu, Aug 30, 2018 at 9:59 PM Sunil G <sun...@apache.org> wrote: > >> Hi All, >> >> Inline with earlier communication dated 17th July 2018, I would like to >> provide some updates. >> >> We are approaching previously proposed code freeze date (Aug 31). >> >> One of the critical feature Node Attributes feature merge discussion/vote >> is ongoing. Also few other Blocker bugs need a bit more time. With regard >> to this, suggesting to push the feature/code freeze for 2 more weeks to >> accommodate these jiras too. >> >> Proposing Updated changes in plan inline with this: >> Feature freeze date : all features to merge by September 7, 2018. >> Code freeze date : blockers/critical only, no improvements and >> blocker/critical bug-fixes September 14, 2018. >> Release date: September 28, 2018 >> >> If any features in branch which are targeted to 3.2.0, please reply to >> this email thread. >> >> *Here's an updated 3.2.0 feature status:* >> >> 1. Merged & Completed features: >> >> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning workloads >> Initial cut. >> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier >> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity Scheduler. >> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN Service API >> and CLI. >> >> 2. Features close to finish: >> >> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. Merge/Vote >> Ongoing. >> - (Rohith) YARN-5742: Serve aggregated logs of historical apps from >> ATSv2. Patch in progress. >> - (Virajit) HDFS-12615: Router-based HDFS federation. Improvement works. >> - (Steve) S3Guard Phase III, S3a phase V, Support Windows Azure Storage. >> In progress. >> >> 3. Tentative features: >> >> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks challenging to >> be done before Aug 2018. >> - (Eric) YARN-7129: Application Catalog for YARN applications. >> Challenging as more discussions are on-going. >> >> *Summary of 3.2.0 issues status:* >> >> 26 Blocker and Critical issues [1] are open, I am following up with >> owners to get status on each of them to get in by Code Freeze date. >> >> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker, >> Critical) AND resolution = Unresolved AND "Target Version/s" = 3.2.0 ORDER >> BY priority DESC >> >> Thanks, >> Sunil >> >> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <sun...@apache.org> wrote: >> >>> Hi All, >>> >>> Thanks for the feedbacks. Inline with earlier communication dated 17th >>> July 2018, I would like to provide some updates. >>> >>> We are approaching previously proposed feature freeze date (Aug 21, >>> about 7 days from today). >>> If any features in branch which are targeted to 3.2.0, please reply to >>> this email thread. >>> Steve has mentioned about the s3 features which will come close to Code >>> Freeze Date (Aug 31st). >>> >>> *Here's an updated 3.2.0 feature status:* >>> >>> 1. Merged & Completed features: >>> >>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning >>> workloads Initial cut. >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier >>> >>> 2. Features close to finish: >>> >>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. Major patches >>> are all in, only one last >>> patch is in review state. >>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity >>> Scheduler. Close to commit. >>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN Service API >>> and CLI. 2 patches are pending >>> which will be closed by Feature freeze date. >>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps from >>> ATSv2. Patch in progress. >>> - (Virajit) HDFS-12615: Router-based HDFS federation. Improvement works. >>> - (Steve) S3Guard Phase III, S3a phase V, Support Windows Azure Storage. >>> In progress. >>> >>> 3. Tentative features: >>> >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks challenging to >>> be done before Aug 2018. >>> - (Eric) YARN-7129: Application Catalog for YARN applications. >>> Challenging as more discussions are on-going. >>> >>> *Summary of 3.2.0 issues status:* >>> >>> 39 Blocker and Critical issues [1] are open, I am checking with owners >>> to get status on each of them to get in by Code Freeze date. >>> >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker, >>> Critical) AND resolution = Unresolved AND "Target Version/s" = 3.2.0 ORDER >>> BY priority DESC >>> >>> Thanks, >>> Sunil >>> >>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <sun...@apache.org> wrote: >>> >>>> Thanks Subru for the thoughts. >>>> One of the main reason for a major release is to push out critical >>>> features with a faster cadence to the users. If we are pulling more and >>>> more different types of features to a minor release, that branch will >>>> become more destabilized and it may be tough to say that 3.1.2 is stable >>>> that 3.1.1 for eg. We always tend to improve and stabilize features in >>>> subsequent minor release. >>>> For few companies, it makes sense to push out these new features faster >>>> to make a reach to the users. Adding to the point to the backporting >>>> issues, I agree that its a pain and we can workaround that with some git >>>> scripts. If we can make such scripts available to committers, backport will >>>> be seem-less across branches and we can achieve the faster release cadence >>>> also. >>>> >>>> Thoughts? >>>> >>>> - Sunil >>>> >>>> >>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <su...@apache.org> >>>> wrote: >>>> >>>>> Thanks Sunil for volunteering to lead the release effort. I am >>>>> generally >>>>> supportive of a release but -1 on a 3.2 (prefer a 3.1.x) as feel we >>>>> already >>>>> have too many branches to be maintained. I already see many commits >>>>> are in >>>>> different branches with no apparent rationale, for e.g: 3.1 has commits >>>>> which are absent in 3.0 etc. >>>>> >>>>> Additionally AFAIK 3.x has not been deployed in any major production >>>>> setting so the cost of adding features should be minimal. >>>>> >>>>> Thoughts? >>>>> >>>>> -Subru >>>>> >>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <sun...@apache.org> wrote: >>>>> >>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts. >>>>> > >>>>> > Yes, important changes and features are much needed, hence we will be >>>>> > keeping the door open for them as possible. Also considering few more >>>>> > offline requests from other folks, I think extending the timeframe by >>>>> > couple of weeks makes sense (including a second RC buffer) and this >>>>> should >>>>> > ideally help us to ship this by September itself. >>>>> > >>>>> > Revised dates (I will be updating same in Roadmap wiki as well) >>>>> > >>>>> > - Feature freeze date : all features to merge by August 21, 2018. >>>>> > >>>>> > - Code freeze date : blockers/critical only, no improvements and non >>>>> > blocker/critical >>>>> > >>>>> > bug-fixes August 31, 2018. >>>>> > >>>>> > - Release date: September 15, 2018 >>>>> > >>>>> > Thank Eric and Zian, I think Wangda has already answered your >>>>> questions. >>>>> > >>>>> > Thanks >>>>> > Sunil >>>>> > >>>>> > >>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <wheele...@gmail.com> >>>>> wrote: >>>>> > >>>>> > > Thanks Sunil for volunteering to be RM of 3.2 release, +1 for that. >>>>> > > >>>>> > > To concerns from Steve, >>>>> > > >>>>> > > It is a good idea to keep the door open to get important changes / >>>>> > > features in before cutoff. I would prefer to keep the proposed >>>>> release >>>>> > date >>>>> > > to make sure things can happen earlier instead of last minute and >>>>> we all >>>>> > > know that releases are always get delayed :). I'm also fine if we >>>>> want >>>>> > get >>>>> > > another several weeks time. >>>>> > > >>>>> > > Regarding of 3.3 release, I would suggest doing that before >>>>> thanksgiving. >>>>> > > Do you think is it good or too early / late? >>>>> > > >>>>> > > Eric, >>>>> > > >>>>> > > The YARN-8220 will be replaced by YARN-8135, if YARN-8135 can get >>>>> merged >>>>> > > in time, we probably not need the YARN-8220. >>>>> > > >>>>> > > Sunil, >>>>> > > >>>>> > > Could u update https://cwiki.apache.org/confluence/display/HADOOP/ >>>>> > Roadmap >>>>> > > with proposed plan as well? We can fill feature list first before >>>>> getting >>>>> > > consensus of time. >>>>> > > >>>>> > > Thanks, >>>>> > > Wangda >>>>> > > >>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri >>>>> <fab...@cloudera.com.invalid >>>>> > > >>>>> > > wrote: >>>>> > > >>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran < >>>>> ste...@hortonworks.com> >>>>> > >> wrote: >>>>> > >> >>>>> > >> > >>>>> > >> > >>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <sun...@apache.org<mailto: >>>>> > >> > sun...@apache.org>> wrote: >>>>> > >> > >>>>> > >> > I would also would like to take this opportunity to come up >>>>> with a >>>>> > >> detailed >>>>> > >> > plan. >>>>> > >> > >>>>> > >> > - Feature freeze date : all features should be merged by August >>>>> 10, >>>>> > >> 2018. >>>>> > >> > >>>>> > >> > >>>>> > >> > >>>>> > >> > <snip> >>>>> > >> >>>>> > >> > >>>>> > >> > Please let me know if I missed any features targeted to 3.2 per >>>>> this >>>>> > >> > >>>>> > >> > >>>>> > >> > Well there these big todo lists for S3 & S3Guard. >>>>> > >> > >>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15226 >>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15220 >>>>> > >> > >>>>> > >> > >>>>> > >> > There's a bigger bit of work coming on for Azure Datalake Gen 2 >>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15407 >>>>> > >> > >>>>> > >> > I don't think this is quite ready yet, I've been doing work on >>>>> it, but >>>>> > >> if >>>>> > >> > we have a 3 week deadline, I'm going to expect some timely >>>>> reviews on >>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15546 >>>>> > >> > >>>>> > >> > I've uprated that to a blocker feature; will review the S3 & >>>>> S3Guard >>>>> > >> JIRAs >>>>> > >> > to see which of those are blocking. Then there are some pressing >>>>> > "guave, >>>>> > >> > java 9 prep" >>>>> > >> > >>>>> > >> > >>>>> > >> I can help with this part if you like. >>>>> > >> >>>>> > >> >>>>> > >> >>>>> > >> > >>>>> > >> > >>>>> > >> > >>>>> > >> > timeline. I would like to volunteer myself as release manager >>>>> of 3.2.0 >>>>> > >> > release. >>>>> > >> > >>>>> > >> > >>>>> > >> > well volunteered! >>>>> > >> > >>>>> > >> > >>>>> > >> > >>>>> > >> Yes, thank you for stepping up. >>>>> > >> >>>>> > >> >>>>> > >> > >>>>> > >> > I think this raises a good q: what timetable should we have for >>>>> the >>>>> > >> 3.2. & >>>>> > >> > 3.3 releases; if we do want a faster cadence, then having the >>>>> outline >>>>> > >> time >>>>> > >> > from the 3.2 to the 3.3 release means that there's less concern >>>>> about >>>>> > >> > things not making the 3.2 dealine >>>>> > >> > >>>>> > >> > -Steve >>>>> > >> > >>>>> > >> > >>>>> > >> Good idea to mitigate the short deadline. >>>>> > >> >>>>> > >> -AF >>>>> > >> >>>>> > > >>>>> > >>>>> >>>>
