I assume the quiet over the holiday means we agreed to move forward without taking HADOOP-14439 into 2.8.2. There is a new release building (docker based) issue could be related to HADOOP-14474 where we removed oracle java 7 installer due to recent download address/contract change by Oracle. The build refuse to work - report as JAVA_HOME issue, but hard coded my local java home in create-release or Dockerfile doesn't help so we may need to add java 7 installation back (no matter Oracle JDK 7 or openJDK 7). Filed HADOOP-14842 with more details to track as blocker for 2.8.2.
Thanks, Junping ________________________________________ From: Junping Du <j...@hortonworks.com> Sent: Friday, September 1, 2017 12:37 PM To: larry mccay; Steve Loughran Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Apache Hadoop 2.8.2 Release Plan This issue (HADOOP-14439) is out of my radar given it is marked as Minor priority. If my understanding is correct, here is a trade-off between security and backward compatibility. IMO, priority of security is generally higher than backward compatibility especially 2.8.0 is still non-production release. I think we should skip this for 2.8.2 in case it doesn't break compatibility from 2.7.x. Thoughts? Thanks, Junping ________________________________________ From: larry mccay <lmc...@apache.org> Sent: Friday, September 1, 2017 10:55 AM To: Steve Loughran Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Apache Hadoop 2.8.2 Release Plan If we do "fix" this in 2.8.2 we should seriously consider not doing so in 3.0. This is a very poor practice. I can see an argument for backward compatibility in 2.8.x line though. On Fri, Sep 1, 2017 at 1:41 PM, Steve Loughran <ste...@hortonworks.com> wrote: > One thing we need to consider is > > HADOOP-14439: regression: secret stripping from S3x URIs breaks some > downstream code > > Hadoop 2.8 has a best-effort attempt to strip out secrets from the > toString() value of an s3a or s3n path where someone has embedded them in > the URI; this has caused problems in some uses, specifically: when people > use secrets this way (bad) and assume that you can round trip paths to > string and back > > Should we fix this? If so, Hadoop 2.8.2 is the time to do it > > > > On 1 Sep 2017, at 11:14, Junping Du <j...@hortonworks.com> wrote: > > > > HADOOP-14814 get committed and HADOOP-9747 get push out to 2.8.3, so we > are clean on blocker/critical issues now. > > I finish practice of going through JACC report and no more incompatible > public API changes get found between 2.8.2 and 2.7.4. Also I check commit > history and fixed 10+ commits which are missing from branch-2.8.2 for some > reason. So, the current branch-2.8.2 should be good to go for RC stage, and > I will kick off our first RC tomorrow. > > In the meanwhile, please don't land any commits to branch-2.8.2 since > now. If some issues really belong to blocker, please ping me on the JIRA > before doing any commits. branch-2.8 is still open for landing. Thanks for > your cooperation! > > > > > > Thanks, > > > > Junping > > > > ________________________________________ > > From: Junping Du <j...@hortonworks.com> > > Sent: Wednesday, August 30, 2017 12:35 AM > > To: Brahma Reddy Battula; common-...@hadoop.apache.org; > hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; > yarn-...@hadoop.apache.org > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > Thanks Brahma for comment on this thread. To be clear, I always update > branch version just before RC kicking off. > > > > For 2.8.2 release, I don't have plan to involve big top or other > third-party test tools. As always, we will rely on test/verify efforts from > community especially from large deployed production cluster - as far as I > know, there are already several companies. like: Yahoo!, Alibaba, etc. > already deploy 2.8 release in large production clusters for months which > give me more confidence on 2.8.2. > > > > > > Here is more update on 2.8.2 release: > > > > Blocker issues: > > > > - A new blocker YARN-7076 get reported and fixed by Jian He through > last weekend. > > > > - Another new blocker - HADOOP-14814 get identified from my latest > jdiff run against 2.7.4. The simple fix on an incompatible API change > should get commit soon. > > > > > > Critical issues: > > > > - YARN-7083 already get committed. Thanks Jason for reporting the > issue and delivering the fix. > > > > - YARN-6091 get push out from 2.8.2 as issue is not a regression and > pending for a while. > > > > - Daryn is actively working on HADOOP-9747 for a while, and the > patch are getting close to be committed. However, according to Daryn, the > patch seems to cause some regression in some corner cases in secured > environment (Storm auto tgt, etc.). May need some additional watch/review > on this JIRA's fixes. > > > > > > > > My monitoring JACC report between 2.8.2 and 2.7.4 will get finish > tomorrow. If everything is going smoothly, I am planning to kick off RC0 > around holiday (this weekend). > > > > > > > > Thanks, > > > > > > > > Junping > > > > > > > > ________________________________ > > From: Brahma Reddy Battula <brahmareddy.batt...@hotmail.com> > > Sent: Tuesday, August 29, 2017 8:42 AM > > To: Junping Du; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > > > Hi All > > > > Update on 2.8.2 release status > > we are down to 3 critical issues ( YARN-6091,YARN-7083,HADOOP-9747),all > are patch available and closer to commit. > > Junping is closing tracking this. > > > > Todo: > > > > 1) Update pom.xml ..? currently it's with 2.8.3 > > https://github.com/apache/hadoop/blob/branch-2.8.2/pom.xml#L21 > > <https://github.com/apache/hadoop/blob/branch-2.8.2/pom.xml#L21>2) Wiki > is outdated, need to update the wiki..? > > 3) As this is going to stable release,are we planing enable Big top for > 2.8.2 testing Or Dynamometer testing (anybody from linked-in can help)..? > > > > @Junping Du<mailto:j...@hortonworks.com>,Please correct me,if I am wrong. > > > > > > --Brahma Reddy Battula > > ________________________________ > > From: Junping Du <j...@hortonworks.com> > > Sent: Monday, August 7, 2017 2:44 PM > > To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > Hello community, > > Here is a quick update on status for 2.8.2: > > - We are 0 blockers now! > > - Still 9 critical issues, 8 of them are Patch Available and with > actively working. > > For details of pending blocker/critical issues, please refer: > https://s.apache.org/JM5x > > Issue Navigator - ASF JIRA<https://s.apache.org/JM5x> > > s.apache.org > > Linked Applications. Loading… Dashboards > > > > > > > > > > I am planning to cut off first RC in week of Aug. 21st to give these > critical issues a bit more time (~2 weeks) to get fixed. Let's working > towards first production GA release of Apache Hadoop 2.8 - let me know if > you have any thoughts or comments. > > > > Cheers, > > > > Junping > > ________________________________________ > > From: Junping Du <j...@hortonworks.com> > > Sent: Monday, July 24, 2017 1:41 PM > > To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org > > Subject: Re: > > > > I have done the change. > > > > All committers, > > > > 2.8.2 release is supposed to be a stable/production release for > branch-2.8. For commits to go for 2.8.2 release (only important and low > risk bug fixes), please commit to trunk, branch-2, branch-2.8 and > branch-2.8.2. For unimportant or high risk bug fixes/improvements, please > commit to branch-2.8 (trunk/branch-2) only and mark JIRA fixed as 2.8.3. > Thanks for your cooperation! > > > > > > Thanks, > > > > > > Junping > > > > > > ________________________________ > > From: Junping Du > > Sent: Monday, July 24, 2017 10:36 AM > > To: Brahma Reddy Battula; Vinod Kumar Vavilapalli > > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; > humbed...@apache.org > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > > > Nice catch, Brahma. Actually, this is not supposed to be happen as we > all should know the patch should firstly get landed on branch-2.8 (as well > as trunk, branch-2) before landed on branch-2.8.2. Anyway, we will always > check JIRAs claim to fixed in a specific release version with commits > actually landing on the releasing branch before kicking off RC. So, I am > not too worry about this mistaking behaviors as it happens in every > releases. > > > > If no other concerns, I will do the branch update in next 30 minutes. > > > > > > Thanks, > > > > > > Junping > > > > > > ________________________________ > > From: Brahma Reddy Battula <brahmareddy.batt...@hotmail.com> > > Sent: Sunday, July 23, 2017 1:50 AM > > To: Junping Du; Vinod Kumar Vavilapalli > > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; > humbed...@apache.org > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > > > > > Just executed the "git log branch-2.8 ^branch-2.8.2" found two commits > are missed (HDFS-8312 and HADOOP-13867 ) in branch-2.8.I just pushed this > two commits.Hope we'll not miss any commits which present in only in > branch-2.8.2. > > > > ________________________________ > > From: Junping Du <j...@hortonworks.com> > > Sent: Saturday, July 22, 2017 5:57 AM > > To: Vinod Kumar Vavilapalli > > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; > humbed...@apache.org > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > Already get back from Daniel who is from ASF INFRA team, I plan to do > following operations on next Monday morning: > > 1. Drop current branch-2.8.2 and recut branch-2.8.2 from branch-2.8 > > 2. Drop abandoned branch-2.8.1 and rename branch-2.8.1-private to > branch-2.8.1 where we just released 2.8.1 from. > > I will also adjust fix version on all affected JIRA accordingly. > > > > If you have any concerns on above operations, please raise it before the > end of this Sunday (7/23). > > > > > > Thanks, > > > > Junping > > > > ________________________________________ > > From: Junping Du <j...@hortonworks.com> > > Sent: Friday, July 21, 2017 2:29 PM > > To: Vinod Kumar Vavilapalli > > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > Make sense, just raise: https://issues.apache.org/ > jira/browse/INFRA-14669 > > [INFRA-14669] Delete several abandoned branches - ASF JIRA< > https://issues.apache.org/jira/browse/INFRA-14669> > > issues.apache.org > > Linked Applications. Loading… Dashboards > > > > > > > > > > Thanks, > > > > Junping > > ________________________________________ > > From: Vinod Kumar Vavilapalli <vino...@apache.org> > > Sent: Friday, July 21, 2017 12:31 PM > > To: Junping Du > > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe > > Subject: Re: Apache Hadoop 2.8.2 Release Plan > > > > Junping, > > > > If we are looking at a month, I’d not rebranch branch-2.8.2 right now > given how these things go. We can just continue to commit on branch-2.8 for > now. > > > > I also think we should just follow up with ASF INFRA and clean up the > branches > > - Delete branch-2.8.2 so that we can recreate it afresh a little later. > > - branch-2.8.1 is also stale and it should be deleted. > branch-2.8.1-private should be renamed to branch-2.8.1 > > > > Thanks > > +Vinod > > > >> On Jul 21, 2017, at 11:23 AM, Junping Du <j...@hortonworks.com> wrote: > >> > >> Thanks for suggestions, Jason and Kihwal! > >> +1 on releasing 2.8.2 on latest branch-2.8 too. Practically, if > branch-2.8.2 cannot be abandoned/replaced (suspect all branches are > read-only now), I will manually merge all commits that not landed on 2.8.2 > yet. > >> > >> Thanks, > >> > >> Junping > >> ________________________________________ > >> From: Jason Lowe <jl...@yahoo-inc.com.INVALID> > >> Sent: Friday, July 21, 2017 8:17 AM > >> To: Kihwal Lee; Junping Du; common-...@hadoop.apache.org; > hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; > yarn-...@hadoop.apache.org > >> Subject: Re: Apache Hadoop 2.8.2 Release Plan > >> > >> +1 to base the 2.8.2 release off of the more recent activity on > branch-2.8. Because branch-2.8.2 was cut so long ago it is missing a lot > of fixes that are in branch-2.8. There also are a lot of JIRAs that claim > they are fixed in 2.8.2 but are not in branch-2.8.2. Having the 2.8.2 > release be based on recent activity in branch-2.8 would solve both of these > issues, and we'd only need to move the handful of JIRAs that have marked > themselves correctly as fixed in 2.8.3 to be fixed in 2.8.2. > >> > >> Jason > >> > >> > >> On Friday, July 21, 2017 10:01 AM, Kihwal Lee > <kih...@yahoo-inc.com.INVALID> wrote: > >> > >> > >> Thanks for driving the next 2.8 release, Junping. While I was > committing a blocker for 2.7.4, I noticed some of the jiras are back-ported > to 2.7, but missing in branch-2.8.2. Perhaps it is safer and easier to > simply rebranch 2.8.2. > >> Thanks,Kihwal > >> > >> On Thursday, July 20, 2017, 3:32:16 PM CDT, Junping Du < > j...@hortonworks.com> wrote: > >> > >> Hi all, > >> Per Vinod's previous email, we just announce Apache Hadoop 2.8.1 get > released today which is a special security release. Now, we should work > towards 2.8.2 release which aim for production deployment. The focus > obviously is to fix blocker/critical issues [2], bug-fixes and *no* > features / improvements. We currently have 13 blocker/critical issues, and > 10 of them are Patch Available. > >> > >> I plan to cut an RC in a month - target for releasing before end of > Aug., to give enough time for outstanding blocker / critical issues. Will > start moving out any tickets that are not blockers and/or won't fit the > timeline. For progress of releasing effort, please refer our release wiki > [2]. > >> > >> Please share thoughts if you have any. Thanks! > >> > >> Thanks, > >> > >> Junping > >> > >> [1] 2.8.2 release Blockers/Criticals: https://s.apache.org/JM5x > > Issue Navigator - ASF JIRA<https://s.apache.org/JM5x> > > s.apache.org > > Linked Applications. Loading… Dashboards > > > > > > > >> [2] 2.8 Release wiki: https://cwiki.apache.org/ > confluence/display/HADOOP/Hadoop+2.8+Release > > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki. > apache.org/confluence/display/HADOOP/Hadoop+2.8+Release> > > cwiki.apache.org > > Release schedule. It has been over 1 year since last minor release > 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per > discussions on community threads ... > > > > > > > > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki. > apache.org/confluence/display/HADOOP/Hadoop+2.8+Release> > > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki. > apache.org/confluence/display/HADOOP/Hadoop+2.8+Release> > > cwiki.apache.org > > Release schedule. It has been over 1 year since last minor release > 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per > discussions on community threads ... > > > > > > > > cwiki.apache.org > > Release schedule. It has been over 1 year since last minor release > 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per > discussions on community threads ... > > > > > > > >> > >> ________________________________________ > >> From: Vinod Kumar Vavilapalli <vino...@apache.org> > >> Sent: Thursday, July 20, 2017 1:05 PM > >> To: gene...@hadoop.apache.org > >> Subject: [ANNOUNCE] Apache Hadoop 2.8.1 is released > >> > >> Hi all, > >> > >> The Apache Hadoop PMC has released version 2.8.1. You can get it from > this page: http://hadoop.apache.org/releases.html#Download > > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download> > > hadoop.apache.org > > To verify Hadoop releases using GPG: Download the release > hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file > hadoop-X.Y.Z-src.tar ... > > > > > > > > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download> > > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download> > > hadoop.apache.org > > To verify Hadoop releases using GPG: Download the release > hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file > hadoop-X.Y.Z-src.tar ... > > > > > > > > hadoop.apache.org > > To verify Hadoop releases using GPG: Download the release > hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file > hadoop-X.Y.Z-src.tar ... > > > > > > > >> This is a security release in the 2.8.0 release line. It consists of > 2.8.0 plus security fixes. Users on 2.8.0 are encouraged to upgrade to > 2.8.1. > >> > >> Please note that 2.8.x release line continues to be not yet ready for > production use. Critical issues are being ironed out via testing and > downstream adoption. Production users should wait for a subsequent release > in the 2.8.x line. > >> > >> Thanks > >> +Vinod > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > >> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > >> > >> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > >> > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
