Thanks Robert and Haibo for quickly correcting same. Sigh, I somehow missed one file while committing the change. Sorry for the trouble.
- Sunil On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter <rkan...@cloudera.com> wrote: > Looks like there's two that weren't updated: > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT" . -r > --include=pom.xml > ./hadoop-project/pom.xml: > <hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version> > ./pom.xml: <hadoop.version>3.2.0-SNAPSHOT</hadoop.version> > > I've just pushed in an addendum commit to fix those. > In the future, please make sure to do a sanity compile when updating poms. > > thanks > - Robert > > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri <fab...@cloudera.com.invalid> > wrote: > >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in the >> top-level pom.xml? >> >> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sun...@apache.org> wrote: >> >> > Hi All >> > >> > As mentioned in earlier mail, I have cut branch-3.2 and reset trunk to >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all necessary >> > patches are pulled into branch-3.2. >> > >> > Thank You >> > - Sunil >> > >> > >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sun...@apache.org> wrote: >> > >> > > Hi All >> > > >> > > We are now down to the last Blocker and HADOOP-15407 is merged to >> trunk. >> > > Thanks for the support. >> > > >> > > *Plan for RC* >> > > 3.2 branch cut and reset trunk : *25th Tuesday* >> > > RC0 for 3.2: *28th Friday* >> > > >> > > Thank You >> > > Sunil >> > > >> > > >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sun...@apache.org> wrote: >> > > >> > >> Hi All >> > >> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all of you for >> > >> helping in this. I am following up on these tickets, once its closed >> we >> > >> will cut the 3.2 branch. >> > >> >> > >> Thanks >> > >> Sunil Govindan >> > >> >> > >> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <sun...@apache.org> wrote: >> > >> >> > >>> Hi All, >> > >>> >> > >>> Inline with the original 3.2 communication proposal dated 17th July >> > >>> 2018, I would like to provide more updates. >> > >>> >> > >>> We are approaching previously proposed code freeze date (September >> 14, >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and point >> > existing >> > >>> trunk to 3.3 if there are no issues. >> > >>> >> > >>> *Current Release Plan:* >> > >>> Feature freeze date : all features to merge by September 7, 2018. >> > >>> Code freeze date : blockers/critical only, no improvements and >> > >>> blocker/critical bug-fixes September 14, 2018. >> > >>> Release date: September 28, 2018 >> > >>> >> > >>> If any critical/blocker tickets which are targeted to 3.2.0, we >> need to >> > >>> backport to 3.2 post branch cut. >> > >>> >> > >>> Here's an updated 3.2.0 feature status: >> > >>> >> > >>> 1. Merged & Completed features: >> > >>> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning >> > >>> workloads Initial cut. >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity >> > >>> Scheduler. >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN Service >> > API >> > >>> and CLI. >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation. Improvement >> works. >> > >>> >> > >>> 2. Features close to finish: >> > >>> >> > >>> - (Steve) S3Guard Phase III. Close to commit. >> > >>> - (Steve) S3a phase V. Close to commit. >> > >>> - (Steve) Support Windows Azure Storage. Close to commit. >> > >>> >> > >>> 3. Tentative/Cancelled features for 3.2: >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps from >> > >>> ATSv2. Patch in progress. >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks >> challenging to >> > >>> be done before Aug 2018. >> > >>> - (Eric) YARN-7129: Application Catalog for YARN applications. >> > >>> Challenging as more discussions are on-going. >> > >>> >> > >>> *Summary of 3.2.0 issues status:* >> > >>> 19 Blocker and Critical issues [1] are open, I am following up with >> > >>> owners to get status on each of them to get in by Code Freeze date. >> > >>> >> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in >> > (Blocker, >> > >>> Critical) AND resolution = Unresolved AND "Target Version/s" = 3.2.0 >> > ORDER >> > >>> BY priority DESC >> > >>> >> > >>> Thanks, >> > >>> Sunil >> > >>> >> > >>> >> > >>> >> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <sun...@apache.org> wrote: >> > >>> >> > >>>> Hi All, >> > >>>> >> > >>>> Inline with earlier communication dated 17th July 2018, I would >> like >> > to >> > >>>> provide some updates. >> > >>>> >> > >>>> We are approaching previously proposed code freeze date (Aug 31). >> > >>>> >> > >>>> One of the critical feature Node Attributes feature merge >> > >>>> discussion/vote is ongoing. Also few other Blocker bugs need a bit >> > more >> > >>>> time. With regard to this, suggesting to push the feature/code >> freeze >> > for 2 >> > >>>> more weeks to accommodate these jiras too. >> > >>>> >> > >>>> Proposing Updated changes in plan inline with this: >> > >>>> Feature freeze date : all features to merge by September 7, 2018. >> > >>>> Code freeze date : blockers/critical only, no improvements and >> > >>>> blocker/critical bug-fixes September 14, 2018. >> > >>>> Release date: September 28, 2018 >> > >>>> >> > >>>> If any features in branch which are targeted to 3.2.0, please >> reply to >> > >>>> this email thread. >> > >>>> >> > >>>> *Here's an updated 3.2.0 feature status:* >> > >>>> >> > >>>> 1. Merged & Completed features: >> > >>>> >> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning >> > >>>> workloads Initial cut. >> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier >> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity >> > >>>> Scheduler. >> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN >> Service >> > >>>> API and CLI. >> > >>>> >> > >>>> 2. Features close to finish: >> > >>>> >> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. >> Merge/Vote >> > >>>> Ongoing. >> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps from >> > >>>> ATSv2. Patch in progress. >> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation. Improvement >> > works. >> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support Windows Azure >> > >>>> Storage. In progress. >> > >>>> >> > >>>> 3. Tentative features: >> > >>>> >> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks >> challenging >> > to >> > >>>> be done before Aug 2018. >> > >>>> - (Eric) YARN-7129: Application Catalog for YARN applications. >> > >>>> Challenging as more discussions are on-going. >> > >>>> >> > >>>> *Summary of 3.2.0 issues status:* >> > >>>> >> > >>>> 26 Blocker and Critical issues [1] are open, I am following up with >> > >>>> owners to get status on each of them to get in by Code Freeze date. >> > >>>> >> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in >> > >>>> (Blocker, Critical) AND resolution = Unresolved AND "Target >> > Version/s" = >> > >>>> 3.2.0 ORDER BY priority DESC >> > >>>> >> > >>>> Thanks, >> > >>>> Sunil >> > >>>> >> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <sun...@apache.org> >> wrote: >> > >>>> >> > >>>>> Hi All, >> > >>>>> >> > >>>>> Thanks for the feedbacks. Inline with earlier communication dated >> > 17th >> > >>>>> July 2018, I would like to provide some updates. >> > >>>>> >> > >>>>> We are approaching previously proposed feature freeze date (Aug >> 21, >> > >>>>> about 7 days from today). >> > >>>>> If any features in branch which are targeted to 3.2.0, please >> reply >> > to >> > >>>>> this email thread. >> > >>>>> Steve has mentioned about the s3 features which will come close to >> > >>>>> Code Freeze Date (Aug 31st). >> > >>>>> >> > >>>>> *Here's an updated 3.2.0 feature status:* >> > >>>>> >> > >>>>> 1. Merged & Completed features: >> > >>>>> >> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for DeepLearning >> > >>>>> workloads Initial cut. >> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier >> > >>>>> >> > >>>>> 2. Features close to finish: >> > >>>>> >> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN. Major >> > >>>>> patches are all in, only one last >> > >>>>> patch is in review state. >> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in Capacity >> > >>>>> Scheduler. Close to commit. >> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via YARN >> Service >> > >>>>> API and CLI. 2 patches are pending >> > >>>>> which will be closed by Feature freeze date. >> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of historical apps >> from >> > >>>>> ATSv2. Patch in progress. >> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation. Improvement >> > >>>>> works. >> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support Windows Azure >> > >>>>> Storage. In progress. >> > >>>>> >> > >>>>> 3. Tentative features: >> > >>>>> >> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks >> challenging >> > >>>>> to be done before Aug 2018. >> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN applications. >> > >>>>> Challenging as more discussions are on-going. >> > >>>>> >> > >>>>> *Summary of 3.2.0 issues status:* >> > >>>>> >> > >>>>> 39 Blocker and Critical issues [1] are open, I am checking with >> > owners >> > >>>>> to get status on each of them to get in by Code Freeze date. >> > >>>>> >> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in >> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND "Target >> > Version/s" = >> > >>>>> 3.2.0 ORDER BY priority DESC >> > >>>>> >> > >>>>> Thanks, >> > >>>>> Sunil >> > >>>>> >> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <sun...@apache.org> >> wrote: >> > >>>>> >> > >>>>>> Thanks Subru for the thoughts. >> > >>>>>> One of the main reason for a major release is to push out >> critical >> > >>>>>> features with a faster cadence to the users. If we are pulling >> more >> > and >> > >>>>>> more different types of features to a minor release, that branch >> > will >> > >>>>>> become more destabilized and it may be tough to say that 3.1.2 is >> > stable >> > >>>>>> that 3.1.1 for eg. We always tend to improve and stabilize >> features >> > in >> > >>>>>> subsequent minor release. >> > >>>>>> For few companies, it makes sense to push out these new features >> > >>>>>> faster to make a reach to the users. Adding to the point to the >> > backporting >> > >>>>>> issues, I agree that its a pain and we can workaround that with >> > some git >> > >>>>>> scripts. If we can make such scripts available to committers, >> > backport will >> > >>>>>> be seem-less across branches and we can achieve the faster >> release >> > cadence >> > >>>>>> also. >> > >>>>>> >> > >>>>>> Thoughts? >> > >>>>>> >> > >>>>>> - Sunil >> > >>>>>> >> > >>>>>> >> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <su...@apache.org >> > >> > >>>>>> wrote: >> > >>>>>> >> > >>>>>>> Thanks Sunil for volunteering to lead the release effort. I am >> > >>>>>>> generally >> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a 3.1.x) as >> feel we >> > >>>>>>> already >> > >>>>>>> have too many branches to be maintained. I already see many >> commits >> > >>>>>>> are in >> > >>>>>>> different branches with no apparent rationale, for e.g: 3.1 has >> > >>>>>>> commits >> > >>>>>>> which are absent in 3.0 etc. >> > >>>>>>> >> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any major >> > production >> > >>>>>>> setting so the cost of adding features should be minimal. >> > >>>>>>> >> > >>>>>>> Thoughts? >> > >>>>>>> >> > >>>>>>> -Subru >> > >>>>>>> >> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <sun...@apache.org> >> > wrote: >> > >>>>>>> >> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts. >> > >>>>>>> > >> > >>>>>>> > Yes, important changes and features are much needed, hence we >> > will >> > >>>>>>> be >> > >>>>>>> > keeping the door open for them as possible. Also considering >> few >> > >>>>>>> more >> > >>>>>>> > offline requests from other folks, I think extending the >> > timeframe >> > >>>>>>> by >> > >>>>>>> > couple of weeks makes sense (including a second RC buffer) and >> > >>>>>>> this should >> > >>>>>>> > ideally help us to ship this by September itself. >> > >>>>>>> > >> > >>>>>>> > Revised dates (I will be updating same in Roadmap wiki as >> well) >> > >>>>>>> > >> > >>>>>>> > - Feature freeze date : all features to merge by August 21, >> 2018. >> > >>>>>>> > >> > >>>>>>> > - Code freeze date : blockers/critical only, no improvements >> and >> > >>>>>>> non >> > >>>>>>> > blocker/critical >> > >>>>>>> > >> > >>>>>>> > bug-fixes August 31, 2018. >> > >>>>>>> > >> > >>>>>>> > - Release date: September 15, 2018 >> > >>>>>>> > >> > >>>>>>> > Thank Eric and Zian, I think Wangda has already answered your >> > >>>>>>> questions. >> > >>>>>>> > >> > >>>>>>> > Thanks >> > >>>>>>> > Sunil >> > >>>>>>> > >> > >>>>>>> > >> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan < >> wheele...@gmail.com >> > > >> > >>>>>>> wrote: >> > >>>>>>> > >> > >>>>>>> > > Thanks Sunil for volunteering to be RM of 3.2 release, +1 >> for >> > >>>>>>> that. >> > >>>>>>> > > >> > >>>>>>> > > To concerns from Steve, >> > >>>>>>> > > >> > >>>>>>> > > It is a good idea to keep the door open to get important >> > changes >> > >>>>>>> / >> > >>>>>>> > > features in before cutoff. I would prefer to keep the >> proposed >> > >>>>>>> release >> > >>>>>>> > date >> > >>>>>>> > > to make sure things can happen earlier instead of last >> minute >> > >>>>>>> and we all >> > >>>>>>> > > know that releases are always get delayed :). I'm also fine >> if >> > >>>>>>> we want >> > >>>>>>> > get >> > >>>>>>> > > another several weeks time. >> > >>>>>>> > > >> > >>>>>>> > > Regarding of 3.3 release, I would suggest doing that before >> > >>>>>>> thanksgiving. >> > >>>>>>> > > Do you think is it good or too early / late? >> > >>>>>>> > > >> > >>>>>>> > > Eric, >> > >>>>>>> > > >> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135, if YARN-8135 >> can >> > >>>>>>> get merged >> > >>>>>>> > > in time, we probably not need the YARN-8220. >> > >>>>>>> > > >> > >>>>>>> > > Sunil, >> > >>>>>>> > > >> > >>>>>>> > > Could u update >> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/ >> > >>>>>>> > Roadmap >> > >>>>>>> > > with proposed plan as well? We can fill feature list first >> > >>>>>>> before getting >> > >>>>>>> > > consensus of time. >> > >>>>>>> > > >> > >>>>>>> > > Thanks, >> > >>>>>>> > > Wangda >> > >>>>>>> > > >> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri >> > >>>>>>> <fab...@cloudera.com.invalid >> > >>>>>>> > > >> > >>>>>>> > > wrote: >> > >>>>>>> > > >> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran < >> > >>>>>>> ste...@hortonworks.com> >> > >>>>>>> > >> wrote: >> > >>>>>>> > >> >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <sun...@apache.org >> > <mailto: >> > >>>>>>> > >> > sun...@apache.org>> wrote: >> > >>>>>>> > >> > >> > >>>>>>> > >> > I would also would like to take this opportunity to come >> up >> > >>>>>>> with a >> > >>>>>>> > >> detailed >> > >>>>>>> > >> > plan. >> > >>>>>>> > >> > >> > >>>>>>> > >> > - Feature freeze date : all features should be merged by >> > >>>>>>> August 10, >> > >>>>>>> > >> 2018. >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > <snip> >> > >>>>>>> > >> >> > >>>>>>> > >> > >> > >>>>>>> > >> > Please let me know if I missed any features targeted to >> 3.2 >> > >>>>>>> per this >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > Well there these big todo lists for S3 & S3Guard. >> > >>>>>>> > >> > >> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15226 >> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15220 >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > There's a bigger bit of work coming on for Azure Datalake >> > Gen >> > >>>>>>> 2 >> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15407 >> > >>>>>>> > >> > >> > >>>>>>> > >> > I don't think this is quite ready yet, I've been doing >> work >> > >>>>>>> on it, but >> > >>>>>>> > >> if >> > >>>>>>> > >> > we have a 3 week deadline, I'm going to expect some >> timely >> > >>>>>>> reviews on >> > >>>>>>> > >> > https://issues.apache.org/jira/browse/HADOOP-15546 >> > >>>>>>> > >> > >> > >>>>>>> > >> > I've uprated that to a blocker feature; will review the >> S3 & >> > >>>>>>> S3Guard >> > >>>>>>> > >> JIRAs >> > >>>>>>> > >> > to see which of those are blocking. Then there are some >> > >>>>>>> pressing >> > >>>>>>> > "guave, >> > >>>>>>> > >> > java 9 prep" >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> I can help with this part if you like. >> > >>>>>>> > >> >> > >>>>>>> > >> >> > >>>>>>> > >> >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > timeline. I would like to volunteer myself as release >> > manager >> > >>>>>>> of 3.2.0 >> > >>>>>>> > >> > release. >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > well volunteered! >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> Yes, thank you for stepping up. >> > >>>>>>> > >> >> > >>>>>>> > >> >> > >>>>>>> > >> > >> > >>>>>>> > >> > I think this raises a good q: what timetable should we >> have >> > >>>>>>> for the >> > >>>>>>> > >> 3.2. & >> > >>>>>>> > >> > 3.3 releases; if we do want a faster cadence, then having >> > the >> > >>>>>>> outline >> > >>>>>>> > >> time >> > >>>>>>> > >> > from the 3.2 to the 3.3 release means that there's less >> > >>>>>>> concern about >> > >>>>>>> > >> > things not making the 3.2 dealine >> > >>>>>>> > >> > >> > >>>>>>> > >> > -Steve >> > >>>>>>> > >> > >> > >>>>>>> > >> > >> > >>>>>>> > >> Good idea to mitigate the short deadline. >> > >>>>>>> > >> >> > >>>>>>> > >> -AF >> > >>>>>>> > >> >> > >>>>>>> > > >> > >>>>>>> > >> > >>>>>>> >> > >>>>>> >> > >> >
