Hi folks,
I would like to give you a quick update on 2.8.2 release status:
- First release candidate (RC0) is published over the last weekend, but several
docker container blockers (bugs, documents, etc.)
are reported so we decided to cancel the RC0 for vote.
- New coming release blockers (for docker container support) are YARN-7034
(just committed), YARN-6623, YARN-6930 and YARN-7230.
Shane, Miklos and Varun are actively working on this. Appreciate the effort
here!
- I will kick off new release candidate (RC1) once these blockers are resolved.
To all committers, branch-2.8.2 is still open for blocker/critical issues
landing, but for major/minor/trivial issues, please commit to branch-2.8 and
marked the fixed version as 2.8.3.
Thanks all for heads up. Have a good weekend!
Thanks,
Junping
________________________________________
From: Junping Du <j...@hortonworks.com>
Sent: Tuesday, September 5, 2017 2:57 PM
To: larry mccay; Steve Loughran
Cc: common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan
I assume the quiet over the holiday means we agreed to move forward without
taking HADOOP-14439 into 2.8.2.
There is a new release building (docker based) issue could be related to
HADOOP-14474 where we removed oracle java 7 installer due to recent download
address/contract change by Oracle. The build refuse to work - report as
JAVA_HOME issue, but hard coded my local java home in create-release or
Dockerfile doesn't help so we may need to add java 7 installation back (no
matter Oracle JDK 7 or openJDK 7).
Filed HADOOP-14842 with more details to track as blocker for 2.8.2.
Thanks,
Junping
________________________________________
From: Junping Du <j...@hortonworks.com>
Sent: Friday, September 1, 2017 12:37 PM
To: larry mccay; Steve Loughran
Cc: common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan
This issue (HADOOP-14439) is out of my radar given it is marked as Minor
priority. If my understanding is correct, here is a trade-off between security
and backward compatibility. IMO, priority of security is generally higher than
backward compatibility especially 2.8.0 is still non-production release.
I think we should skip this for 2.8.2 in case it doesn't break compatibility
from 2.7.x. Thoughts?
Thanks,
Junping
________________________________________
From: larry mccay <lmc...@apache.org>
Sent: Friday, September 1, 2017 10:55 AM
To: Steve Loughran
Cc: common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan
If we do "fix" this in 2.8.2 we should seriously consider not doing so in
3.0.
This is a very poor practice.
I can see an argument for backward compatibility in 2.8.x line though.
On Fri, Sep 1, 2017 at 1:41 PM, Steve Loughran <ste...@hortonworks.com>
wrote:
> One thing we need to consider is
>
> HADOOP-14439: regression: secret stripping from S3x URIs breaks some
> downstream code
>
> Hadoop 2.8 has a best-effort attempt to strip out secrets from the
> toString() value of an s3a or s3n path where someone has embedded them in
> the URI; this has caused problems in some uses, specifically: when people
> use secrets this way (bad) and assume that you can round trip paths to
> string and back
>
> Should we fix this? If so, Hadoop 2.8.2 is the time to do it
>
>
> > On 1 Sep 2017, at 11:14, Junping Du <j...@hortonworks.com> wrote:
> >
> > HADOOP-14814 get committed and HADOOP-9747 get push out to 2.8.3, so we
> are clean on blocker/critical issues now.
> > I finish practice of going through JACC report and no more incompatible
> public API changes get found between 2.8.2 and 2.7.4. Also I check commit
> history and fixed 10+ commits which are missing from branch-2.8.2 for some
> reason. So, the current branch-2.8.2 should be good to go for RC stage, and
> I will kick off our first RC tomorrow.
> > In the meanwhile, please don't land any commits to branch-2.8.2 since
> now. If some issues really belong to blocker, please ping me on the JIRA
> before doing any commits. branch-2.8 is still open for landing. Thanks for
> your cooperation!
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Junping Du <j...@hortonworks.com>
> > Sent: Wednesday, August 30, 2017 12:35 AM
> > To: Brahma Reddy Battula; common-...@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org;
> yarn-...@hadoop.apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Thanks Brahma for comment on this thread. To be clear, I always update
> branch version just before RC kicking off.
> >
> > For 2.8.2 release, I don't have plan to involve big top or other
> third-party test tools. As always, we will rely on test/verify efforts from
> community especially from large deployed production cluster - as far as I
> know, there are already several companies. like: Yahoo!, Alibaba, etc.
> already deploy 2.8 release in large production clusters for months which
> give me more confidence on 2.8.2.
> >
> >
> > Here is more update on 2.8.2 release:
> >
> > Blocker issues:
> >
> > - A new blocker YARN-7076 get reported and fixed by Jian He through
> last weekend.
> >
> > - Another new blocker - HADOOP-14814 get identified from my latest
> jdiff run against 2.7.4. The simple fix on an incompatible API change
> should get commit soon.
> >
> >
> > Critical issues:
> >
> > - YARN-7083 already get committed. Thanks Jason for reporting the
> issue and delivering the fix.
> >
> > - YARN-6091 get push out from 2.8.2 as issue is not a regression and
> pending for a while.
> >
> > - Daryn is actively working on HADOOP-9747 for a while, and the
> patch are getting close to be committed. However, according to Daryn, the
> patch seems to cause some regression in some corner cases in secured
> environment (Storm auto tgt, etc.). May need some additional watch/review
> on this JIRA's fixes.
> >
> >
> >
> > My monitoring JACC report between 2.8.2 and 2.7.4 will get finish
> tomorrow. If everything is going smoothly, I am planning to kick off RC0
> around holiday (this weekend).
> >
> >
> >
> > Thanks,
> >
> >
> >
> > Junping
> >
> >
> >
> > ________________________________
> > From: Brahma Reddy Battula <brahmareddy.batt...@hotmail.com>
> > Sent: Tuesday, August 29, 2017 8:42 AM
> > To: Junping Du; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> >
> > Hi All
> >
> > Update on 2.8.2 release status
> > we are down to 3 critical issues ( YARN-6091,YARN-7083,HADOOP-9747),all
> are patch available and closer to commit.
> > Junping is closing tracking this.
> >
> > Todo:
> >
> > 1) Update pom.xml ..? currently it's with 2.8.3
> > https://github.com/apache/hadoop/blob/branch-2.8.2/pom.xml#L21
> > <https://github.com/apache/hadoop/blob/branch-2.8.2/pom.xml#L21>2) Wiki
> is outdated, need to update the wiki..?
> > 3) As this is going to stable release,are we planing enable Big top for
> 2.8.2 testing Or Dynamometer testing (anybody from linked-in can help)..?
> >
> > @Junping Du<mailto:j...@hortonworks.com>,Please correct me,if I am wrong.
> >
> >
> > --Brahma Reddy Battula
> > ________________________________
> > From: Junping Du <j...@hortonworks.com>
> > Sent: Monday, August 7, 2017 2:44 PM
> > To: common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Hello community,
> > Here is a quick update on status for 2.8.2:
> > - We are 0 blockers now!
> > - Still 9 critical issues, 8 of them are Patch Available and with
> actively working.
> > For details of pending blocker/critical issues, please refer:
> https://s.apache.org/JM5x
> > Issue Navigator - ASF JIRA<https://s.apache.org/JM5x>
> > s.apache.org
> > Linked Applications. Loading… Dashboards
> >
> >
> >
> >
> > I am planning to cut off first RC in week of Aug. 21st to give these
> critical issues a bit more time (~2 weeks) to get fixed. Let's working
> towards first production GA release of Apache Hadoop 2.8 - let me know if
> you have any thoughts or comments.
> >
> > Cheers,
> >
> > Junping
> > ________________________________________
> > From: Junping Du <j...@hortonworks.com>
> > Sent: Monday, July 24, 2017 1:41 PM
> > To: common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org
> > Subject: Re:
> >
> > I have done the change.
> >
> > All committers,
> >
> > 2.8.2 release is supposed to be a stable/production release for
> branch-2.8. For commits to go for 2.8.2 release (only important and low
> risk bug fixes), please commit to trunk, branch-2, branch-2.8 and
> branch-2.8.2. For unimportant or high risk bug fixes/improvements, please
> commit to branch-2.8 (trunk/branch-2) only and mark JIRA fixed as 2.8.3.
> Thanks for your cooperation!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Junping Du
> > Sent: Monday, July 24, 2017 10:36 AM
> > To: Brahma Reddy Battula; Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe;
> humbed...@apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> >
> > Nice catch, Brahma. Actually, this is not supposed to be happen as we
> all should know the patch should firstly get landed on branch-2.8 (as well
> as trunk, branch-2) before landed on branch-2.8.2. Anyway, we will always
> check JIRAs claim to fixed in a specific release version with commits
> actually landing on the releasing branch before kicking off RC. So, I am
> not too worry about this mistaking behaviors as it happens in every
> releases.
> >
> > If no other concerns, I will do the branch update in next 30 minutes.
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Brahma Reddy Battula <brahmareddy.batt...@hotmail.com>
> > Sent: Sunday, July 23, 2017 1:50 AM
> > To: Junping Du; Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe;
> humbed...@apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> >
> >
> > Just executed the "git log branch-2.8 ^branch-2.8.2" found two commits
> are missed (HDFS-8312 and HADOOP-13867 ) in branch-2.8.I just pushed this
> two commits.Hope we'll not miss any commits which present in only in
> branch-2.8.2.
> >
> > ________________________________
> > From: Junping Du <j...@hortonworks.com>
> > Sent: Saturday, July 22, 2017 5:57 AM
> > To: Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe;
> humbed...@apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Already get back from Daniel who is from ASF INFRA team, I plan to do
> following operations on next Monday morning:
> > 1. Drop current branch-2.8.2 and recut branch-2.8.2 from branch-2.8
> > 2. Drop abandoned branch-2.8.1 and rename branch-2.8.1-private to
> branch-2.8.1 where we just released 2.8.1 from.
> > I will also adjust fix version on all affected JIRA accordingly.
> >
> > If you have any concerns on above operations, please raise it before the
> end of this Sunday (7/23).
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Junping Du <j...@hortonworks.com>
> > Sent: Friday, July 21, 2017 2:29 PM
> > To: Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Make sense, just raise: https://issues.apache.org/
> jira/browse/INFRA-14669
> > [INFRA-14669] Delete several abandoned branches - ASF JIRA<
> https://issues.apache.org/jira/browse/INFRA-14669>
> > issues.apache.org
> > Linked Applications. Loading… Dashboards
> >
> >
> >
> >
> > Thanks,
> >
> > Junping
> > ________________________________________
> > From: Vinod Kumar Vavilapalli <vino...@apache.org>
> > Sent: Friday, July 21, 2017 12:31 PM
> > To: Junping Du
> > Cc: Kihwal Lee; common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Junping,
> >
> > If we are looking at a month, I’d not rebranch branch-2.8.2 right now
> given how these things go. We can just continue to commit on branch-2.8 for
> now.
> >
> > I also think we should just follow up with ASF INFRA and clean up the
> branches
> > - Delete branch-2.8.2 so that we can recreate it afresh a little later.
> > - branch-2.8.1 is also stale and it should be deleted.
> branch-2.8.1-private should be renamed to branch-2.8.1
> >
> > Thanks
> > +Vinod
> >
> >> On Jul 21, 2017, at 11:23 AM, Junping Du <j...@hortonworks.com> wrote:
> >>
> >> Thanks for suggestions, Jason and Kihwal!
> >> +1 on releasing 2.8.2 on latest branch-2.8 too. Practically, if
> branch-2.8.2 cannot be abandoned/replaced (suspect all branches are
> read-only now), I will manually merge all commits that not landed on 2.8.2
> yet.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: Jason Lowe <jl...@yahoo-inc.com.INVALID>
> >> Sent: Friday, July 21, 2017 8:17 AM
> >> To: Kihwal Lee; Junping Du; common-...@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org;
> yarn-...@hadoop.apache.org
> >> Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >>
> >> +1 to base the 2.8.2 release off of the more recent activity on
> branch-2.8. Because branch-2.8.2 was cut so long ago it is missing a lot
> of fixes that are in branch-2.8. There also are a lot of JIRAs that claim
> they are fixed in 2.8.2 but are not in branch-2.8.2. Having the 2.8.2
> release be based on recent activity in branch-2.8 would solve both of these
> issues, and we'd only need to move the handful of JIRAs that have marked
> themselves correctly as fixed in 2.8.3 to be fixed in 2.8.2.
> >>
> >> Jason
> >>
> >>
> >> On Friday, July 21, 2017 10:01 AM, Kihwal Lee
> <kih...@yahoo-inc.com.INVALID> wrote:
> >>
> >>
> >> Thanks for driving the next 2.8 release, Junping. While I was
> committing a blocker for 2.7.4, I noticed some of the jiras are back-ported
> to 2.7, but missing in branch-2.8.2. Perhaps it is safer and easier to
> simply rebranch 2.8.2.
> >> Thanks,Kihwal
> >>
> >> On Thursday, July 20, 2017, 3:32:16 PM CDT, Junping Du <
> j...@hortonworks.com> wrote:
> >>
> >> Hi all,
> >> Per Vinod's previous email, we just announce Apache Hadoop 2.8.1 get
> released today which is a special security release. Now, we should work
> towards 2.8.2 release which aim for production deployment. The focus
> obviously is to fix blocker/critical issues [2], bug-fixes and *no*
> features / improvements. We currently have 13 blocker/critical issues, and
> 10 of them are Patch Available.
> >>
> >> I plan to cut an RC in a month - target for releasing before end of
> Aug., to give enough time for outstanding blocker / critical issues. Will
> start moving out any tickets that are not blockers and/or won't fit the
> timeline. For progress of releasing effort, please refer our release wiki
> [2].
> >>
> >> Please share thoughts if you have any. Thanks!
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >> [1] 2.8.2 release Blockers/Criticals: https://s.apache.org/JM5x
> > Issue Navigator - ASF JIRA<https://s.apache.org/JM5x>
> > s.apache.org
> > Linked Applications. Loading… Dashboards
> >
> >
> >
> >> [2] 2.8 Release wiki: https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release
> > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki.
> apache.org/confluence/display/HADOOP/Hadoop+2.8+Release>
> > cwiki.apache.org
> > Release schedule. It has been over 1 year since last minor release
> 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per
> discussions on community threads ...
> >
> >
> >
> > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki.
> apache.org/confluence/display/HADOOP/Hadoop+2.8+Release>
> > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki.
> apache.org/confluence/display/HADOOP/Hadoop+2.8+Release>
> > cwiki.apache.org
> > Release schedule. It has been over 1 year since last minor release
> 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per
> discussions on community threads ...
> >
> >
> >
> > cwiki.apache.org
> > Release schedule. It has been over 1 year since last minor release
> 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per
> discussions on community threads ...
> >
> >
> >
> >>
> >> ________________________________________
> >> From: Vinod Kumar Vavilapalli <vino...@apache.org>
> >> Sent: Thursday, July 20, 2017 1:05 PM
> >> To: gene...@hadoop.apache.org
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.1 is released
> >>
> >> Hi all,
> >>
> >> The Apache Hadoop PMC has released version 2.8.1. You can get it from
> this page: http://hadoop.apache.org/releases.html#Download
> > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download>
> > hadoop.apache.org
> > To verify Hadoop releases using GPG: Download the release
> hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file
> hadoop-X.Y.Z-src.tar ...
> >
> >
> >
> > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download>
> > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download>
> > hadoop.apache.org
> > To verify Hadoop releases using GPG: Download the release
> hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file
> hadoop-X.Y.Z-src.tar ...
> >
> >
> >
> > hadoop.apache.org
> > To verify Hadoop releases using GPG: Download the release
> hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file
> hadoop-X.Y.Z-src.tar ...
> >
> >
> >
> >> This is a security release in the 2.8.0 release line. It consists of
> 2.8.0 plus security fixes. Users on 2.8.0 are encouraged to upgrade to
> 2.8.1.
> >>
> >> Please note that 2.8.x release line continues to be not yet ready for
> production use. Critical issues are being ironed out via testing and
> downstream adoption. Production users should wait for a subsequent release
> in the 2.8.x line.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> >> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
