Re: [VOTE] Moving Ozone to a separated Apache project

[Zhankun Tang]

+1. I believe that Ozone becomes TLP will help it to grow faster.

BR,
Zhankun

On Mon, 28 Sep 2020 at 15:41, Surendra Singh Lilhore <
surendralilh...@apache.org> wrote:

> +1
>
> Thanks
> Surendra
>
>
> On Mon, 28 Sep, 2020, 12:19 pm Akira Ajisaka,  wrote:
>
> > +1
> >
> > Thanks,
> > Akira
> >
> > On Fri, Sep 25, 2020 at 3:00 PM Elek, Marton  wrote:
> > >
> > > Hi all,
> > >
> > > Thank you for all the feedback and requests,
> > >
> > > As we discussed in the previous thread(s) [1], Ozone is proposed to be
> a
> > > separated Apache Top Level Project (TLP)
> > >
> > > The proposal with all the details, motivation and history is here:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/HADOOP/Ozone+Hadoop+subproject+to+Apache+TLP+proposal
> > >
> > > This voting runs for 7 days and will be concluded at 2nd of October,
> 6AM
> > > GMT.
> > >
> > > Thanks,
> > > Marton Elek
> > >
> > > [1]:
> > >
> >
> https://lists.apache.org/thread.html/rc6c79463330b3e993e24a564c6817aca1d290f186a1206c43ff0436a%40%3Chdfs-dev.hadoop.apache.org%3E
> > >
> > > -
> > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >
>

Re: [DISCUSS] Shade guava into hadoop-thirdparty

[Zhankun Tang]

Thanks, Wei-Chiu for the proposal. +1.

On Mon, 6 Apr 2020 at 20:17, Ayush Saxena  wrote:

> +1
>
> -Ayush
>
> > On 05-Apr-2020, at 12:43 AM, Wei-Chiu Chuang  wrote:
> >
> > Hi Hadoop devs,
> >
> > I spent a good part of the past 7 months working with a dozen of
> colleagues
> > to update the guava version in Cloudera's software (that includes Hadoop,
> > HBase, Spark, Hive, Cloudera Manager ... more than 20+ projects)
> >
> > After 7 months, I finally came to a conclusion: Update to Hadoop 3.3 /
> > 3.2.1 / 3.1.3, even if you just go from Hadoop 3.0/ 3.1.0 is going to be
> > really hard because of guava. Because of Guava, the amount of work to
> > certify a minor release update is almost equivalent to a major release
> > update.
> >
> > That is because:
> > (1) Going from guava 11 to guava 27 is a big jump. There are several
> > incompatible API changes in many places. Too bad the Google developers
> are
> > not sympathetic about its users.
> > (2) guava is used in all Hadoop jars. Not just Hadoop servers but also
> > client jars and Hadoop common libs.
> > (3) The Hadoop library is used in practically all software at Cloudera.
> >
> > Here is my proposal:
> > (1) shade guava into hadoop-thirdparty, relocate the classpath to
> > org.hadoop.thirdparty.com.google.common.*
> > (2) make a hadoop-thirdparty 1.1.0 release.
> > (3) update existing references to guava to the relocated path. There are
> > more than 2k imports that need an update.
> > (4) release Hadoop 3.3.1 / 3.2.2 that contains this change.
> >
> > In this way, we will be able to update guava in Hadoop in the future
> > without disrupting Hadoop applications.
> >
> > Note: HBase already did this and this guava update project would have
> been
> > much more difficult if HBase didn't do so.
> >
> > Thoughts? Other options include
> > (1) force downstream applications to migrate to Hadoop client artifacts
> as
> > listed here
> >
> https://hadoop.apache.org/docs/r3.1.1/hadoop-project-dist/hadoop-common/DownstreamDev.html
> > but
> > that's nearly impossible.
> > (2) Migrate Guava to Java APIs. I suppose this is a big project and I
> can't
> > estimate how much work it's going to be.
> >
> > Weichiu
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>

Re: [ANNOUNCE] New Apache Hadoop Committer - Siyao Meng

[Zhankun Tang]

Welcome onboard. Siyao!

BR,
Zhankun

Xiaoyu Yao 于2020年3月21日 周六上午1:24写道：

> It's my pleasure to announce that Siyao Meng has been elected as committer
> on the Apache Hadoop project recognizing his continued contributions to the
> project.
>
> Please join me in congratulating him.
>
> Congratulations & Welcome aboard Siyao!
>
> Xiaoyu Yao
> (On behalf of the Hadoop PMC)
>

Re: [VOTE] EOL Hadoop branch-2.8

[Zhankun Tang]

Thanks, Wei-Chiu. +1.

BR,
Zhankun

On Wed, 4 Mar 2020 at 08:03, Wilfred Spiegelenburg
 wrote:

> +1
>
> Wilfred
>
> > On 3 Mar 2020, at 05:48, Wei-Chiu Chuang  wrote:
> >
> > I am sorry I forgot to start a VOTE thread.
> >
> > This is the "official" vote thread to mark branch-2.8 End of Life. This
> is
> > based on the following thread and the tracking jira (HADOOP-16880
> > ).
> >
> > This vote will run for 7 days and conclude on March 9th (Mon) 11am PST.
> >
> > Please feel free to share your thoughts.
> >
> > Thanks,
> > Weichiu
> >
> > On Mon, Feb 24, 2020 at 10:28 AM Wei-Chiu Chuang 
> > wrote:
> >
> >> Looking at the EOL policy wiki:
> >>
> https://cwiki.apache.org/confluence/display/HADOOP/EOL+%28End-of-life%29+Release+Branches
> >>
> >> The Hadoop community can still elect to make security update for EOL'ed
> >> releases.
> >>
> >> I think the EOL is to give more clarity to downstream applications (such
> >> as HBase) the guidance of which Hadoop release lines are still active.
> >> Additionally, I don't think it is sustainable to maintain 6 concurrent
> >> release lines in this big project, which is why I wanted to start this
> >> discussion.
> >>
> >> Thoughts?
> >>
> >> On Mon, Feb 24, 2020 at 10:22 AM Sunil Govindan 
> wrote:
> >>
> >>> Hi Wei-Chiu
> >>>
> >>> Extremely sorry for the late reply here.
> >>> Cud u pls help to add more clarity on defining what will happen for
> >>> branch-2.8 when we call EOL.
> >>> Does this mean that, no more release coming out from this branch, or
> some
> >>> more additional guidelines?
> >>>
> >>> - Sunil
> >>>
> >>>
> >>> On Mon, Feb 24, 2020 at 11:47 PM Wei-Chiu Chuang
> >>>  wrote:
> >>>
>  This thread has been running for 7 days and no -1.
> 
>  Don't think we've established a formal EOL process, but to publicize
> the
>  EOL, I am going to file a jira, update the wiki and post the
> >>> announcement
>  to general@ and user@
> 
>  On Wed, Feb 19, 2020 at 1:40 PM Dinesh Chitlangia <
> >>> dineshc@gmail.com>
>  wrote:
> 
> > Thanks Wei-Chiu for initiating this.
> >
> > +1 for 2.8 EOL.
> >
> > On Tue, Feb 18, 2020 at 10:48 PM Akira Ajisaka 
> > wrote:
> >
> >> Thanks Wei-Chiu for starting the discussion,
> >>
> >> +1 for the EoL.
> >>
> >> -Akira
> >>
> >> On Tue, Feb 18, 2020 at 4:59 PM Ayush Saxena 
>  wrote:
> >>
> >>> Thanx Wei-Chiu for initiating this
> >>> +1 for marking 2.8 EOL
> >>>
> >>> -Ayush
> >>>
>  On 17-Feb-2020, at 11:14 PM, Wei-Chiu Chuang <
> >>> weic...@apache.org>
> >> wrote:
> 
>  The last Hadoop 2.8.x release, 2.8.5, was GA on September 15th
>  2018.
> 
>  It's been 17 months since the release and the community by and
>  large
> >> have
>  moved up to 2.9/2.10/3.x.
> 
>  With Hadoop 3.3.0 over the horizon, is it time to start the EOL
> >>> discussion
>  and reduce the number of active branches?
> >>>
> >>>
> >>> -
> >>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> >>> For additional commands, e-mail:
> >>> common-dev-h...@hadoop.apache.org
> >>>
> >>>
> >>
> >
> 
> >>>
> >>
>
> Wilfred Spiegelenburg
> Staff Software Engineer
>  
>

Re: [ANNOUNCE] New Apache Hadoop Committer - Stephen O'Donnell

[Zhankun Tang]

Congrats! Stephen!

BR,
Zhankun

On Wed, 4 Mar 2020 at 04:27, Ayush Saxena  wrote:

> Congratulations Stephen!!!
>
> -Ayush
>
> > On 04-Mar-2020, at 1:51 AM, Bharat Viswanadham 
> wrote:
> >
> > Congratulations Stephen!
> >
> > Thanks,
> > Bharat
> >
> >
> >> On Tue, Mar 3, 2020 at 12:12 PM Wei-Chiu Chuang 
> wrote:
> >>
> >> In bcc: general@
> >>
> >> It's my pleasure to announce that Stephen O'Donnell has been elected as
> >> committer on the Apache Hadoop project recognizing his continued
> >> contributions to the
> >> project.
> >>
> >> Please join me in congratulating him.
> >>
> >> Hearty Congratulations & Welcome aboard Stephen!
> >>
> >> Wei-Chiu Chuang
> >> (On behalf of the Hadoop PMC)
> >>
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>

Meeting minutes - Re: Monthly YARN & MR community sync-up (Mandarin) reminder

[Zhankun Tang]

Hi,

We've got 19 peoples joined this meeting from various big internet
companies in China. The discussion was quite active and lasts extra
half-hour.

The common background recently is to improve the online service K8s cluster
utilization with batch jobs. Our key topic today is utilization
improvement solution including YARN+K8s to run both batch and services, and
how to do oversubscription. etc.

See the notes for more details.
https://docs.google.com/document/d/1GY55sXrekVd-aDyRY7uzaX0hMDPyh3T-AL1kUY2TI5M/edit#heading=h.1er20w6ibaxm

BR,
Zhankun


On Thu, 20 Feb 2020 at 11:29, Zhankun Tang  wrote:

> Hi,
> This is a friendly reminder that we have the monthly YARN & MR community
> sync up (Mandarin) today at 2 pm (CST), 10 pm (PST, Feb 19, 2020). Hope to
> see you there!
>
>
> https://docs.google.com/document/d/1GY55sXrekVd-aDyRY7uzaX0hMDPyh3T-AL1kUY2TI5M/edit#heading=h.6wgz1xgh0qde
>
> BR,
> Zhankun
>

Monthly YARN & MR community sync-up (Mandarin) reminder

[Zhankun Tang]

Hi,
This is a friendly reminder that we have the monthly YARN & MR community
sync up (Mandarin) today at 2 pm (CST), 10 pm (PST, Feb 19, 2020). Hope to
see you there!

https://docs.google.com/document/d/1GY55sXrekVd-aDyRY7uzaX0hMDPyh3T-AL1kUY2TI5M/edit#heading=h.6wgz1xgh0qde

BR,
Zhankun

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

[zhankun tang]

Thanks for the efforts! Jonathan!

+1 (non-binding) on RC1.
- Set up a single node cluster with the binary tarball
- Run Spark Pi and PySpark job

BR,
Zhankun

On Mon, 28 Oct 2019 at 23:16, Masatake Iwasaki 
wrote:

> Thanks for putting this up, Jonathan Hung.
>
> +1(non-binding)
>
> * verified signature and checksum.
> * built source tarball by OpenJDK 7 on CentOS 7 with native profile
> enabled.
> * launched pseudo distributed cluster with security configurations.
> * create encryption zone and put files in it.
> * accessed files in encryption zone via httpfs.
> * checked that file names containing '%' or '+' or ';' can be handled by
> webhdfs.
>
> Masatake Iwasaki
>
> On 10/23/19 06:55, Jonathan Hung wrote:
> > Hi folks,
> >
> > This is the second release candidate for the first release of Apache
> Hadoop
> > 2.10 line. It contains 362 fixes/improvements since 2.9 [1]. It includes
> > features such as:
> >
> > - User-defined resource types
> > - Native GPU support as a schedulable resource type
> > - Consistent reads from standby node
> > - Namenode port based selective encryption
> > - Improvements related to rolling upgrade support from 2.x to 3.x
> > - Cost based fair call queue
> >
> > The RC1 artifacts are at:
> http://home.apache.org/~jhung/hadoop-2.10.0-RC1/
> >
> > RC tag is release-2.10.0-RC1.
> >
> > The maven artifacts are hosted here:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1243/
> >
> > My public key is available here:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > The vote will run for 5 weekdays, until Tuesday, October 29 at 3:00 pm
> PDT.
> >
> > Thanks,
> > Jonathan Hung
> >
> > [1]
> >
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >
>
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Zhankun Tang]

Thanks for the effort, Jonathan!

+1 (non-binding) on RC0.
 - Set up a single node cluster with the binary tarball
 - Run Spark Pi and pySpark job

BR,
Zhankun

On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko 
wrote:

> +1 on RC0.
> - Verified signatures
> - Built from sources
> - Ran unit tests for new features
> - Checked artifacts on Nexus, made sure the sources are present.
>
> Thanks
> --Konstantin
>
>
> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung 
> wrote:
>
> > Hi folks,
> >
> > This is the first release candidate for the first release of Apache
> Hadoop
> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> > features such as:
> >
> > - User-defined resource types
> > - Native GPU support as a schedulable resource type
> > - Consistent reads from standby node
> > - Namenode port based selective encryption
> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >
> > The RC0 artifacts are at:
> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >
> > RC tag is release-2.10.0-RC0.
> >
> > The maven artifacts are hosted here:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >
> > My public key is available here:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> > PDT.
> >
> > Thanks,
> > Jonathan Hung
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >
>

[ANNOUNCE] Apache Hadoop 3.1.3 release!

[Zhankun Tang]

Hi folks,

It's a  great honor to announce that the Apache Hadoop community
has released Apache Hadoop 3.1.3.

Apache Hadoop 3.1.3 is the stable maintenance release of Apache Hadoop 3.1
line, which includes 246 fixes and improvements since Hadoop 3.1.2.

Thanks to our community!

*Apache Hadoop 3.1.3 released*: https://hadoop.apache.org/release/3.1.3.html
*Changelog*: https://s.apache.org/dqxla
*Release note*: https://s.apache.org/x3sgj

BR,
Zhankun

Re: VOTE PASSED - Hadoop-3.1.3-RC0 Re: [VOTE] Release Hadoop-3.1.3-RC0

[Zhankun Tang]

Vinod,

Thanks for offering the help.
Yeah. I haven't finished the remaining steps. Due to the permission issue,
I cannot push the artifacts to the hadoop dist repo for now.
And Sunil is helping on this.

I'll ping you if any other issue. Thanks.

BR,
Zhankun


Vinod Kumar Vavilapalli 于2019年10月13日 周日上午1:08写道：

> Zhankun et.al,
>
> Did we close down the rest of the release process for 3.1.3?
>
> I don't yet see this on the site, so asking.
>
> If it's not done yet and you are running into issues, please let me know
> and I can help with the corresponding release management tasks.
>
> Thanks
> +Vinod
>
> On Tue, Sep 24, 2019 at 2:16 PM Zhankun Tang  wrote:
>
> > Hi,
> >
> > Thank you all who spent time to verify and vote 3.1.3 release!
> > I‘m pleased to announce that the vote for 3.1.3 RC0 passed! I'll push the
> > release bits and send out an announcement for 3.1.3 soon.
> >
> > Summary of votes for Hadoop-3.1.3-RC0:
> >
> > 4 binding +1s, from Rohith Sharma K S, Sunil Govindan, Weiwei Yang, Eric
> > Payne
> >
> > 3 non-binding +1s, from Xun Liu, Zac Zhou, Zhankun Tang
> >
> > 1 non-binding with +0s from Masatake Iwasaki
> >
> > and *no -1s*.
> >
> > BR,
> > Zhankun
> >
> > On Tue, 24 Sep 2019 at 15:40, zac yuan  wrote:
> >
> > > Looking forward to 3.1.3 release. Thanks Zhankun for your great effort~
> > >
> > > +1 (non-binding)
> > >
> > > Zac Zhou
> > >
> > > Xun Liu  于2019年9月23日周一 上午11:49写道：
> > >
> > > > +1
> > > >
> > > > Thanks Zhankun for all of your hard work on this release.
> > > >
> > > >
> > > > > On Sep 20, 2019, at 5:34 AM, epa...@apache.org wrote:
> > > > >
> > > > >
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks Zhankun for all of your hard work on this release.
> > > > >
> > > > > I downloaded and built the source and ran it on an insecure
> > multi-node
> > > > pseudo cluster.
> > > > >
> > > > > I performed various YARN manual tests, including creating custom
> > > > resources, creating queue submission ACLs, and queue refreshes.
> > > > >
> > > > > One concern is that preemption does not seem to be working when
> only
> > > the
> > > > custom resources are over the queue capacity, but I don't think this
> is
> > > > something introduced with this release.
> > > > >
> > > > > -Eric
> > > > >
> > > > >
> > > > >
> > > > > On Thursday, September 12, 2019, 3:04:44 AM CDT, Zhankun Tang <
> > > > zt...@apache.org> wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi folks,
> > > > >
> > > > > Thanks to everyone's help on this release. Special thanks to
> Rohith,
> > > > > Wei-Chiu, Akira, Sunil, Wangda!
> > > > >
> > > > > I have created a release candidate (RC0) for Apache Hadoop 3.1.3.
> > > > >
> > > > > The RC release artifacts are available at:
> > > > > http://home.apache.org/~ztang/hadoop-3.1.3-RC0/
> > > > >
> > > > > The maven artifacts are staged at:
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1228/
> > > > >
> > > > > The RC tag in git is here:
> > > > > https://github.com/apache/hadoop/tree/release-3.1.3-RC0
> > > > >
> > > > > And my public key is at:
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > > >
> > > > > *This vote will run for 7 days, ending on Sept.19th at 11:59 pm
> PST.*
> > > > >
> > > > > For the testing, I have run several Spark and distributed shell
> jobs
> > in
> > > > my
> > > > > pseudo cluster.
> > > > >
> > > > > My +1 (non-binding) to start.
> > > > >
> > > > > BR,
> > > > > Zhankun
> > > > >
> > > > > On Wed, 4 Sep 2019 at 15:56, zhankun tang 
> > > wrote:
> > > > >
> > > > >> Hi all,
> > > > >>
> > > > >> Thanks for everyone helping in resolving all the bloc

VOTE PASSED - Hadoop-3.1.3-RC0 Re: [VOTE] Release Hadoop-3.1.3-RC0

[Zhankun Tang]

Hi,

Thank you all who spent time to verify and vote 3.1.3 release!
I‘m pleased to announce that the vote for 3.1.3 RC0 passed! I'll push the
release bits and send out an announcement for 3.1.3 soon.

Summary of votes for Hadoop-3.1.3-RC0:

4 binding +1s, from Rohith Sharma K S, Sunil Govindan, Weiwei Yang, Eric
Payne

3 non-binding +1s, from Xun Liu, Zac Zhou, Zhankun Tang

1 non-binding with +0s from Masatake Iwasaki

and *no -1s*.

BR,
Zhankun

On Tue, 24 Sep 2019 at 15:40, zac yuan  wrote:

> Looking forward to 3.1.3 release. Thanks Zhankun for your great effort~
>
> +1 (non-binding)
>
> Zac Zhou
>
> Xun Liu  于2019年9月23日周一 上午11:49写道：
>
> > +1
> >
> > Thanks Zhankun for all of your hard work on this release.
> >
> >
> > > On Sep 20, 2019, at 5:34 AM, epa...@apache.org wrote:
> > >
> > >
> > >
> > > +1 (binding)
> > >
> > > Thanks Zhankun for all of your hard work on this release.
> > >
> > > I downloaded and built the source and ran it on an insecure multi-node
> > pseudo cluster.
> > >
> > > I performed various YARN manual tests, including creating custom
> > resources, creating queue submission ACLs, and queue refreshes.
> > >
> > > One concern is that preemption does not seem to be working when only
> the
> > custom resources are over the queue capacity, but I don't think this is
> > something introduced with this release.
> > >
> > > -Eric
> > >
> > >
> > >
> > > On Thursday, September 12, 2019, 3:04:44 AM CDT, Zhankun Tang <
> > zt...@apache.org> wrote:
> > >
> > >
> > >
> > >
> > >
> > > Hi folks,
> > >
> > > Thanks to everyone's help on this release. Special thanks to Rohith,
> > > Wei-Chiu, Akira, Sunil, Wangda!
> > >
> > > I have created a release candidate (RC0) for Apache Hadoop 3.1.3.
> > >
> > > The RC release artifacts are available at:
> > > http://home.apache.org/~ztang/hadoop-3.1.3-RC0/
> > >
> > > The maven artifacts are staged at:
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1228/
> > >
> > > The RC tag in git is here:
> > > https://github.com/apache/hadoop/tree/release-3.1.3-RC0
> > >
> > > And my public key is at:
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > >
> > > *This vote will run for 7 days, ending on Sept.19th at 11:59 pm PST.*
> > >
> > > For the testing, I have run several Spark and distributed shell jobs in
> > my
> > > pseudo cluster.
> > >
> > > My +1 (non-binding) to start.
> > >
> > > BR,
> > > Zhankun
> > >
> > > On Wed, 4 Sep 2019 at 15:56, zhankun tang 
> wrote:
> > >
> > >> Hi all,
> > >>
> > >> Thanks for everyone helping in resolving all the blockers targeting
> > Hadoop
> > >> 3.1.3[1]. We've cleaned all the blockers and moved out non-blockers
> > issues
> > >> to 3.1.4.
> > >>
> > >> I'll cut the branch today and call a release vote soon. Thanks!
> > >>
> > >>
> > >> [1]. https://s.apache.org/5hj5i
> > >>
> > >> BR,
> > >> Zhankun
> > >>
> > >>
> > >> On Wed, 21 Aug 2019 at 12:38, Zhankun Tang  wrote:
> > >>
> > >>> Hi folks,
> > >>>
> > >>> We have Apache Hadoop 3.1.2 released on Feb 2019.
> > >>>
> > >>> It's been more than 6 months passed and there're
> > >>>
> > >>> 246 fixes[1]. 2 blocker and 4 critical Issues [2]
> > >>>
> > >>> (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker)
> > >>>
> > >>>
> > >>> I propose my plan to do a maintenance release of 3.1.3 in the next
> few
> > >>> (one or two) weeks.
> > >>>
> > >>> Hadoop 3.1.3 release plan:
> > >>>
> > >>> Code Freezing Date: *25th August 2019 PDT*
> > >>>
> > >>> Release Date: *31th August 2019 PDT*
> > >>>
> > >>>
> > >>> Please feel free to share your insights on this. Thanks!
> > >>>
> > >>>
> > >>> [1] https://s.apache.org/zw8l5
> > >>>
> > >>> [2] https://s.apache.org/fjol5
> > >>>
> > >>>
> > >>> BR,
> > >>>
> > >>> Zhankun
> > >>>
> > >>
> > >
> > > -
> > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >
> > -
> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
> >
> >
>

Re: [VOTE] Release Hadoop-3.1.3-RC0

[zhankun tang]

Hi Masatake,

My bad. I read it wrong. Yeah. You're right. The HDFS-14759 can be
backported to both Branch-3.2 and 3.1.

Let's see if there's any other blocker/critical issue come up, otherwise,
personally I don't prefer to run another RC1 and vote for this minor change.

BR,
Zhankun

On Wed, 18 Sep 2019 at 15:02, Masatake Iwasaki 
wrote:

> Hi Zhankun,
>
>  > Can you please help to provide a branch-3.1 patch for HDFS-14759? Or
> we can
>  > move it to the next release of branch-3.1 since the noisy info is not a
>  > blocker issue to me. Does that make sense?
>
> I tried to cherry-picking the HDFS-14759 on my side.
> Since there was no conflict, I pushed the branch-3.1 to apache repo.
> Could you pull and check it?
>
> Masatake
>
> On 9/18/19 15:49, Zhankun Tang wrote:
> > Hi Masatake,
> >
> > Thanks for helping to verify!
> > I checked that branch-3.2 has the HDFS-14759 committed already.
> > Release-3.2.1-RC0 should have no such issue.
> >
> > For branch-3.1, cherry-pick the same commit has conflicts. I'm confirming
> > if we can fix it or there's a feasible plan to backport the whole
> > Hadoop-15226 (which targeting branch-3.2) to branch-3.1.
> >
> > Can you please help to provide a branch-3.1 patch for HDFS-14759? Or we
> can
> > move it to the next release of branch-3.1 since the noisy info is not a
> > blocker issue to me. Does that make sense?
> >
> >
> > BR,
> > Zhankun
> >
> > On Wed, 18 Sep 2019 at 14:23, Masatake Iwasaki <
> iwasak...@oss.nttdata.co.jp>
> > wrote:
> >
> >> Thanks for putting this up, Zhankun Tang.
> >>
> >> While I was testing the RC0 wich CLI,
> >> noisy INFO message was emitted on every data transfer operation.::
> >>
> >> 2019-09-17 16:00:42,942 INFO sasl.SaslDataTransferClient: SASL
> >> encryption trust check: localHostTrusted = false, remoteHostTrusted =
> false
> >>
> >> The issue was fixed by HDFS-14759.
> >> I think the fix should be backported if we cut RC1.
> >>
> >> Since the fix version of HDFS-14759 is 3.3.0, CR0 of 3.2.1 could have
> >> same issue.
> >>
> >> Regards,
> >> Masatake Iwasaki
> >>
> >> On 9/12/19 17:04, Zhankun Tang wrote:
> >>> Hi folks,
> >>>
> >>> Thanks to everyone's help on this release. Special thanks to Rohith,
> >>> Wei-Chiu, Akira, Sunil, Wangda!
> >>>
> >>> I have created a release candidate (RC0) for Apache Hadoop 3.1.3.
> >>>
> >>> The RC release artifacts are available at:
> >>> http://home.apache.org/~ztang/hadoop-3.1.3-RC0/
> >>>
> >>> The maven artifacts are staged at:
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1228/
> >>>
> >>> The RC tag in git is here:
> >>> https://github.com/apache/hadoop/tree/release-3.1.3-RC0
> >>>
> >>> And my public key is at:
> >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>>
> >>> *This vote will run for 7 days, ending on Sept.19th at 11:59 pm PST.*
> >>>
> >>> For the testing, I have run several Spark and distributed shell jobs in
> >> my
> >>> pseudo cluster.
> >>>
> >>> My +1 (non-binding) to start.
> >>>
> >>> BR,
> >>> Zhankun
> >>>
> >>> On Wed, 4 Sep 2019 at 15:56, zhankun tang 
> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> Thanks for everyone helping in resolving all the blockers targeting
> >> Hadoop
> >>>> 3.1.3[1]. We've cleaned all the blockers and moved out non-blockers
> >> issues
> >>>> to 3.1.4.
> >>>>
> >>>> I'll cut the branch today and call a release vote soon. Thanks!
> >>>>
> >>>>
> >>>> [1]. https://s.apache.org/5hj5i
> >>>>
> >>>> BR,
> >>>> Zhankun
> >>>>
> >>>>
> >>>> On Wed, 21 Aug 2019 at 12:38, Zhankun Tang  wrote:
> >>>>
> >>>>> Hi folks,
> >>>>>
> >>>>> We have Apache Hadoop 3.1.2 released on Feb 2019.
> >>>>>
> >>>>> It's been more than 6 months passed and there're
> >>>>>
> >>>>> 246 fixes[1]. 2 blocker and 4 critical Issues [2]
> >>>>>
> >>>>> (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker)
> >>>>>
> >>>>>
> >>>>> I propose my plan to do a maintenance release of 3.1.3 in the next
> few
> >>>>> (one or two) weeks.
> >>>>>
> >>>>> Hadoop 3.1.3 release plan:
> >>>>>
> >>>>> Code Freezing Date: *25th August 2019 PDT*
> >>>>>
> >>>>> Release Date: *31th August 2019 PDT*
> >>>>>
> >>>>>
> >>>>> Please feel free to share your insights on this. Thanks!
> >>>>>
> >>>>>
> >>>>> [1] https://s.apache.org/zw8l5
> >>>>>
> >>>>> [2] https://s.apache.org/fjol5
> >>>>>
> >>>>>
> >>>>> BR,
> >>>>>
> >>>>> Zhankun
> >>>>>
> >>
>
>

Re: [VOTE] Release Hadoop-3.1.3-RC0

[Zhankun Tang]

Hi Masatake,

Thanks for helping to verify!
I checked that branch-3.2 has the HDFS-14759 committed already.
Release-3.2.1-RC0 should have no such issue.

For branch-3.1, cherry-pick the same commit has conflicts. I'm confirming
if we can fix it or there's a feasible plan to backport the whole
Hadoop-15226 (which targeting branch-3.2) to branch-3.1.

Can you please help to provide a branch-3.1 patch for HDFS-14759? Or we can
move it to the next release of branch-3.1 since the noisy info is not a
blocker issue to me. Does that make sense?


BR,
Zhankun

On Wed, 18 Sep 2019 at 14:23, Masatake Iwasaki 
wrote:

> Thanks for putting this up, Zhankun Tang.
>
> While I was testing the RC0 wich CLI,
> noisy INFO message was emitted on every data transfer operation.::
>
>2019-09-17 16:00:42,942 INFO sasl.SaslDataTransferClient: SASL
> encryption trust check: localHostTrusted = false, remoteHostTrusted = false
>
> The issue was fixed by HDFS-14759.
> I think the fix should be backported if we cut RC1.
>
> Since the fix version of HDFS-14759 is 3.3.0, CR0 of 3.2.1 could have
> same issue.
>
> Regards,
> Masatake Iwasaki
>
> On 9/12/19 17:04, Zhankun Tang wrote:
> > Hi folks,
> >
> > Thanks to everyone's help on this release. Special thanks to Rohith,
> > Wei-Chiu, Akira, Sunil, Wangda!
> >
> > I have created a release candidate (RC0) for Apache Hadoop 3.1.3.
> >
> > The RC release artifacts are available at:
> > http://home.apache.org/~ztang/hadoop-3.1.3-RC0/
> >
> > The maven artifacts are staged at:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1228/
> >
> > The RC tag in git is here:
> > https://github.com/apache/hadoop/tree/release-3.1.3-RC0
> >
> > And my public key is at:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > *This vote will run for 7 days, ending on Sept.19th at 11:59 pm PST.*
> >
> > For the testing, I have run several Spark and distributed shell jobs in
> my
> > pseudo cluster.
> >
> > My +1 (non-binding) to start.
> >
> > BR,
> > Zhankun
> >
> > On Wed, 4 Sep 2019 at 15:56, zhankun tang  wrote:
> >
> >> Hi all,
> >>
> >> Thanks for everyone helping in resolving all the blockers targeting
> Hadoop
> >> 3.1.3[1]. We've cleaned all the blockers and moved out non-blockers
> issues
> >> to 3.1.4.
> >>
> >> I'll cut the branch today and call a release vote soon. Thanks!
> >>
> >>
> >> [1]. https://s.apache.org/5hj5i
> >>
> >> BR,
> >> Zhankun
> >>
> >>
> >> On Wed, 21 Aug 2019 at 12:38, Zhankun Tang  wrote:
> >>
> >>> Hi folks,
> >>>
> >>> We have Apache Hadoop 3.1.2 released on Feb 2019.
> >>>
> >>> It's been more than 6 months passed and there're
> >>>
> >>> 246 fixes[1]. 2 blocker and 4 critical Issues [2]
> >>>
> >>> (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker)
> >>>
> >>>
> >>> I propose my plan to do a maintenance release of 3.1.3 in the next few
> >>> (one or two) weeks.
> >>>
> >>> Hadoop 3.1.3 release plan:
> >>>
> >>> Code Freezing Date: *25th August 2019 PDT*
> >>>
> >>> Release Date: *31th August 2019 PDT*
> >>>
> >>>
> >>> Please feel free to share your insights on this. Thanks!
> >>>
> >>>
> >>> [1] https://s.apache.org/zw8l5
> >>>
> >>> [2] https://s.apache.org/fjol5
> >>>
> >>>
> >>> BR,
> >>>
> >>> Zhankun
> >>>
>
>

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

[zhankun tang]

+1 (non-binding).
Installed and verified it by running several Spark job and DS jobs.

BR,
Zhankun

On Wed, 18 Sep 2019 at 08:05, Naganarasimha Garla <
naganarasimha...@apache.org> wrote:

> Verified the source and the binary tar and the sha512 checksums
> Installed and verified the basic hadoop operations (ran few MR tasks)
>
> +1.
>
> Thanks,
> + Naga
>
> On Wed, Sep 18, 2019 at 1:32 AM Anil Sadineni 
> wrote:
>
> > +1 (non-binding)
> >
> > On Tue, Sep 17, 2019 at 9:55 AM Santosh Marella 
> wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Wed, Sep 11, 2019 at 12:26 AM Rohith Sharma K S <
> > > rohithsharm...@apache.org> wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I have put together a release candidate (RC0) for Apache Hadoop
> 3.2.1.
> > > >
> > > > The RC is available at:
> > > > http://home.apache.org/~rohithsharmaks/hadoop-3.2.1-RC0/
> > > >
> > > > The RC tag in git is release-3.2.1-RC0:
> > > > https://github.com/apache/hadoop/tree/release-3.2.1-RC0
> > > >
> > > >
> > > > The maven artifacts are staged at
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1226/
> > > >
> > > > You can find my public key at:
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > >
> > > > This vote will run for 7 days(5 weekdays), ending on 18th Sept at
> 11:59
> > > pm
> > > > PST.
> > > >
> > > > I have done testing with a pseudo cluster and distributed shell job.
> My
> > > +1
> > > > to start.
> > > >
> > > > Thanks & Regards
> > > > Rohith Sharma K S
> > > >
> > >
> >
> >
> > --
> > Thanks & Regards,
> > Anil Sadineni
> > Solutions Architect, Optlin Inc
> > Ph: 571-438-1974 | www.optlin.com
> >
>

[VOTE] Release Hadoop-3.1.3-RC0

[Zhankun Tang]

Hi folks,

Thanks to everyone's help on this release. Special thanks to Rohith,
Wei-Chiu, Akira, Sunil, Wangda!

I have created a release candidate (RC0) for Apache Hadoop 3.1.3.

The RC release artifacts are available at:
http://home.apache.org/~ztang/hadoop-3.1.3-RC0/

The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1228/

The RC tag in git is here:
https://github.com/apache/hadoop/tree/release-3.1.3-RC0

And my public key is at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

*This vote will run for 7 days, ending on Sept.19th at 11:59 pm PST.*

For the testing, I have run several Spark and distributed shell jobs in my
pseudo cluster.

My +1 (non-binding) to start.

BR,
Zhankun

On Wed, 4 Sep 2019 at 15:56, zhankun tang  wrote:

> Hi all,
>
> Thanks for everyone helping in resolving all the blockers targeting Hadoop
> 3.1.3[1]. We've cleaned all the blockers and moved out non-blockers issues
> to 3.1.4.
>
> I'll cut the branch today and call a release vote soon. Thanks!
>
>
> [1]. https://s.apache.org/5hj5i
>
> BR,
> Zhankun
>
>
> On Wed, 21 Aug 2019 at 12:38, Zhankun Tang  wrote:
>
>> Hi folks,
>>
>> We have Apache Hadoop 3.1.2 released on Feb 2019.
>>
>> It's been more than 6 months passed and there're
>>
>> 246 fixes[1]. 2 blocker and 4 critical Issues [2]
>>
>> (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker)
>>
>>
>> I propose my plan to do a maintenance release of 3.1.3 in the next few
>> (one or two) weeks.
>>
>> Hadoop 3.1.3 release plan:
>>
>> Code Freezing Date: *25th August 2019 PDT*
>>
>> Release Date: *31th August 2019 PDT*
>>
>>
>> Please feel free to share your insights on this. Thanks!
>>
>>
>> [1] https://s.apache.org/zw8l5
>>
>> [2] https://s.apache.org/fjol5
>>
>>
>> BR,
>>
>> Zhankun
>>
>

[jira] [Resolved] (HADOOP-16551) The changelog*.md seems not generated when create-release

[Zhankun Tang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhankun Tang resolved HADOOP-16551.
---
Resolution: Fixed

Close this since revert HADOOP-16061 has fixed it.

> The changelog*.md seems not generated when create-release
> -
>
> Key: HADOOP-16551
> URL: https://issues.apache.org/jira/browse/HADOOP-16551
> Project: Hadoop Common
>  Issue Type: Task
>    Reporter: Zhankun Tang
>Priority: Blocker
>
> Hi,
>  When creating Hadoop 3.1.3 release with "create-release" script, after the 
> mvn site succeeded. But it complains about this and failed:
> {code:java}
> dev-support/bin/create-release --asfrelease --docker --dockercache{code}
> {code:java}
> $ cd /build/source
> $ mv /build/source/target/hadoop-site-3.1.3.tar.gz 
> /build/source/target/artifacts/hadoop-3.1.3-site.tar.gz
> $ cp -p 
> /build/source/hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3/CHANGES*.md
>  /build/source/target/artifacts/CHANGES.md
> cp: cannot stat 
> '/build/source/hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3/CHANGES*.md':
>  No such file or directory
> {code}
> And there's no 3.1.3 release site markdown folder.
> {code:java}
> [ztang@release-vm hadoop]$ ls 
> hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3
> ls: cannot access 
> hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3: No such 
> file or directory
> {code}
> I've checked the HADOOP-14671 but have no idea why this changelog is missing.
> *Update:*
>  Found that the CHANGELOG.md and RELEASENOTES.md are generated but not in 
> directory "3.1.3"
> {code:java}
> [ztang@release-vm hadoop]$ ls 
> hadoop-common-project/hadoop-common/src/site/markdown/release/
> 0.1.0 0.15.2 0.19.2 0.23.2 0.7.2 2.0.1-alpha 2.6.3 3.0.0-alpha3
> 0.10.0 0.15.3 0.2.0 0.23.3 0.8.0 2.0.2-alpha 2.6.4 3.0.0-alpha4
> 0.10.1 0.15.4 0.20.0 0.23.4 0.9.0 2.0.3-alpha 2.6.5 3.0.0-beta1
> 0.1.1 0.16.0 0.20.1 0.23.5 0.9.1 2.0.4-alpha 2.6.6 3.0.1
> 0.11.0 0.16.1 0.20.2 0.23.6 0.9.2 2.0.5-alpha 2.7.0 3.0.3
> 0.11.1 0.16.2 0.20.203.0 0.23.7 1.0.0 2.0.6-alpha 2.7.1 3.1.0
> 0.11.2 0.16.3 0.20.203.1 0.23.8 1.0.1 2.1.0-beta 2.7.2 3.1.1
> 0.12.0 0.16.4 0.20.204.0 0.23.9 1.0.2 2.1.1-beta 2.7.3 3.1.2
> 0.12.1 0.17.0 0.20.205.0 0.24.0 1.0.3 2.2.0 2.7.4 CHANGELOG.md
> 0.12.2 0.17.1 0.20.3 0.3.0 1.0.4 2.2.1 2.7.5 index.md
> 0.12.3 0.17.2 0.2.1 0.3.1 1.1.0 2.3.0 2.8.0 README.md
> 0.13.0 0.17.3 0.21.0 0.3.2 1.1.1 2.4.0 2.8.1 RELEASENOTES.md
> 0.14.0 0.18.0 0.21.1 0.4.0 1.1.2 2.4.1 2.8.2
> 0.14.1 0.18.1 0.22.0 0.5.0 1.1.3 2.5.0 2.8.3
> 0.14.2 0.18.2 0.22.1 0.6.0 1.2.0 2.5.1 2.9.0
> 0.14.3 0.18.3 0.23.0 0.6.1 1.2.1 2.5.2 2.9.1
> 0.14.4 0.18.4 0.23.1 0.6.2 1.2.2 2.6.0 3.0.0
> 0.15.0 0.19.0 0.23.10 0.7.0 1.3.0 2.6.1 3.0.0-alpha1
> 0.15.1 0.19.1 0.23.11 0.7.1 2.0.0-alpha 2.6.2 3.0.0-alpha2{code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-16551) The changelog*.md seems not generated when create-release

[Zhankun Tang (Jira)]

Zhankun Tang created HADOOP-16551:
-

 Summary: The changelog*.md seems not generated when create-release
 Key: HADOOP-16551
 URL: https://issues.apache.org/jira/browse/HADOOP-16551
 Project: Hadoop Common
  Issue Type: Task
Reporter: Zhankun Tang


Hi,
 When creating Hadoop 3.1.3 release with "create-release" script, after the mvn 
site succeeded. But it complains about this and failed:

{code:java}
dev-support/bin/create-release --asfrelease --docker --dockercache{code}
{code:java}
$ cd /build/source
$ mv /build/source/target/hadoop-site-3.1.3.tar.gz 
/build/source/target/artifacts/hadoop-3.1.3-site.tar.gz
$ cp -p 
/build/source/hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3/CHANGES*.md
 /build/source/target/artifacts/CHANGES.md
cp: cannot stat 
'/build/source/hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3/CHANGES*.md':
 No such file or directory
{code}

And there's no 3.1.3 release site markdown folder.
{code:java}
[ztang@release-vm hadoop]$ ls 
hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3
ls: cannot access 
hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.3: No such 
file or directory

{code}
I've checked the HADOOP-14671 but have no idea why this changelog is not 
generated.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-16545) Update the release year to 2019

[Zhankun Tang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhankun Tang resolved HADOOP-16545.
---
Resolution: Duplicate

close this due to duplicated with HADOOP-16025

> Update the release year to 2019
> ---
>
> Key: HADOOP-16545
> URL: https://issues.apache.org/jira/browse/HADOOP-16545
> Project: Hadoop Common
>  Issue Type: Task
>    Reporter: Zhankun Tang
>Priority: Critical
>
> Is doing the release. We need to update the release year from 2018 to 2019.
> {code:java}
> $ find . -name "pom.xml" | xargs grep -n 2018
> ./hadoop-project/pom.xml:34:2018
> {code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-16545) Update the release year to 2019

[Zhankun Tang (Jira)]

Zhankun Tang created HADOOP-16545:
-

 Summary: Update the release year to 2019
 Key: HADOOP-16545
 URL: https://issues.apache.org/jira/browse/HADOOP-16545
 Project: Hadoop Common
  Issue Type: Task
Reporter: Zhankun Tang


Is doing the release. We need to update the release year from 2018 to 2019.

{code:java}
$ find . -name "pom.xml" | xargs grep -n 2018
./hadoop-project/pom.xml:34:2018
{code}




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Hadoop 3.1.3 is ready to cut branch Re: [Release Plan] Hadoop-3.1.3 discussion

[zhankun tang]

Hi all,

Thanks for everyone helping in resolving all the blockers targeting Hadoop
3.1.3[1]. We've cleaned all the blockers and moved out non-blockers issues
to 3.1.4.

I'll cut the branch today and call a release vote soon. Thanks!


[1]. https://s.apache.org/5hj5i

BR,
Zhankun


On Wed, 21 Aug 2019 at 12:38, Zhankun Tang  wrote:

> Hi folks,
>
> We have Apache Hadoop 3.1.2 released on Feb 2019.
>
> It's been more than 6 months passed and there're
>
> 246 fixes[1]. 2 blocker and 4 critical Issues [2]
>
> (As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker)
>
>
> I propose my plan to do a maintenance release of 3.1.3 in the next few
> (one or two) weeks.
>
> Hadoop 3.1.3 release plan:
>
> Code Freezing Date: *25th August 2019 PDT*
>
> Release Date: *31th August 2019 PDT*
>
>
> Please feel free to share your insights on this. Thanks!
>
>
> [1] https://s.apache.org/zw8l5
>
> [2] https://s.apache.org/fjol5
>
>
> BR,
>
> Zhankun
>

Re: [VOTE] Moving Submarine to a separate Apache project proposal

[zhankun tang]

+1

Thanks for Wangda's proposal.

The submarine project is born within Hadoop, but not limited to Hadoop. It
began with a trainer on YARN but it quickly realized that only a trainer is
not enough to meet the AI platform requirements. But now there's no
user-friendly open-source solution covers the whole AI pipeline like data
engineering, training, and serving. And the underlying data infrastructure
itself is also evolving, for instance, many people love k8s. Not mentioning
there're many AI domain problems in this area to be solved.
It's almost for sure that building such an ML platform would utilize
various other open-source components taking ML into consideration initially.

I see submarine grows rapidly towards an enterprise-grade ML platform which
could potentially enable AI ability for data engineer and scientist. This
is an exciting thing for both the community and the industry.

BR,
Zhankun


On Tue, 3 Sep 2019 at 13:34, Xun Liu  wrote:

> +1
>
> Hello everyone, I am a member of the submarine development team.
> I have been contributing to submarine for more than a year.
> I have seen the progress of submarine development very fast.
> In more than a year, there are 9 long-term developers of different
> companies. Contributing,
> submarine cumulative code has more than 200,000 lines of code, is growing
> very fast,
> and is used in the production environment of multiple companies.
>
> In the submarine development group, there are 5 PMCs and 7committer members
> from Hadoop, spark, zeppelin projects.
> They are very familiar with the development process and specifications of
> the apache community,
> and can well grasp the project development progress and project quality.
> So I recommend submarine to be a TLP project directly.
>
> We will continue to contribute to the submarine project. :-)
>
> Xun Liu
> Regards
>
> On Tue, 3 Sep 2019 at 12:01, Devaraj K  wrote:
>
> > +1
> >
> > Thanks Wangda for the proposal.
> > I would like to participate in this project, Please add me also to the
> > project.
> >
> > Regards
> > Devaraj K
> >
> > On Mon, Sep 2, 2019 at 8:50 PM zac yuan  wrote:
> >
> > > +1
> > >
> > > Submarine will be a complete solution for AI service development.  It
> can
> > > take advantage of two best cluster systems: yarn and k8s, which will
> help
> > > more and more people get AI ability. To be a separate Apache project,
> > will
> > > accelerate the procedure of development apparently.
> > >
> > > Look forward to a big success in submarine project~
> > >
> > > 朱林浩  于2019年9月3日周二 上午10:38写道：
> > >
> > > > +1,
> > > > Hopefully, that will become the top project,
> > > >
> > > > I also hope to make more contributions to this project.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > At 2019-09-03 09:26:53, "Naganarasimha Garla" <
> > > naganarasimha...@apache.org>
> > > > wrote:
> > > > >+ 1,
> > > > >I would also like start participate in this project, hope to get
> > > > myself
> > > > >added to the project.
> > > > >
> > > > >Thanks and Regards,
> > > > >+ Naga
> > > > >
> > > > >On Tue, Sep 3, 2019 at 8:35 AM Wangda Tan 
> > wrote:
> > > > >
> > > > >> Hi Sree,
> > > > >>
> > > > >> I put it to the proposal, please let me know what you think:
> > > > >>
> > > > >> The traditional path at Apache would have been to create an
> > incubator
> > > > >> > project, but the code is already being released by Apache and
> most
> > > of
> > > > the
> > > > >> > developers are familiar with Apache rules and guidelines. In
> > > > particular,
> > > > >> > the proposed PMC has 2 Apache TLP PMCs and proposed initial
> > > committers
> > > > >> > have 4 Apache TLP PMCs (from 3 different companies). They will
> > > provide
> > > > >> > oversight and guidance for the developers that are less
> > experienced
> > > in
> > > > >> the
> > > > >> > Apache Way. Therefore, the Submarine project would like to
> propose
> > > > >> becoming
> > > > >> > a Top Level Project at Apache.
> > > > >> >
> > > > >>
> > > > >> To me, putting to TLP has mostly pros, it is an easier process
> (same
> > > as
> > > > ORC
> > > > >> spin-off from Hive), much less overhead to both dev community and
> > > Apache
> > > > >> side.
> > > > >>
> > > > >> Thanks,
> > > > >> Wangda
> > > > >>
> > > > >> On Sun, Sep 1, 2019 at 2:04 PM Sree Vaddi <
> sree_at_ch...@yahoo.com>
> > > > wrote:
> > > > >>
> > > > >> > +1 to move submarine to separate apache project.
> > > > >> >
> > > > >> > It is not clear in the proposal, if submarine majority voted to
> > move
> > > > to a
> > > > >> > seperate apache project,
> > > > >> > will it go through the incubation and TLP (top level project)
> > later
> > > ?
> > > > >> > 1. Incubation
> > > > >> > pros and cons
> > > > >> > efforts towards making it a TLP
> > > > >> >
> > > > >> > 2. direct TLP
> > > > >> >
> > > > >> >
> > > > >> > Thank you.
> > > > >> > /Sree
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Saturday, August 31, 2019, 10:19:06 PM 

[Release Plan] Hadoop-3.1.3 discussion

[Zhankun Tang]

Hi folks,

We have Apache Hadoop 3.1.2 released on Feb 2019.

It's been more than 6 months passed and there're

246 fixes[1]. 2 blocker and 4 critical Issues [2]

(As Wei-Chiu Chuang mentioned, HDFS-13596 will be another blocker)


I propose my plan to do a maintenance release of 3.1.3 in the next few (one
or two) weeks.

Hadoop 3.1.3 release plan:

Code Freezing Date: *25th August 2019 PDT*

Release Date: *31th August 2019 PDT*


Please feel free to share your insights on this. Thanks!


[1] https://s.apache.org/zw8l5

[2] https://s.apache.org/fjol5


BR,

Zhankun

Re: [DISCUSS] Hadoop 2019 Release Planning

[zhankun tang]

Thanks Wangda for bring this up!

I ran the submarine 0.2.0 release before with a lot of help from folks
especially Sunil. :D
And this time I would like to help to release the 3.1.4. Thanks!

BR,
Zhankun

Hui Fei 于2019年8月16日 周五下午7:19写道：

> Hi Wangda,
> Thanks for bringing this up!
> Looking forward to see HDFS 3.x is widely used，but RollingUpgrade is a
> problem.
> Hope commiters watch and review these issues, Thanks
> https://issues.apache.org/jira/browse/HDFS-13596
> https://issues.apache.org/jira/browse/HDFS-14396
>
> Wangda Tan  于2019年8月10日周六 上午10:59写道：
>
> > Hi all,
> >
> > Hope this email finds you well
> >
> > I want to hear your thoughts about what should be the release plan for
> > 2019.
> >
> > In 2018, we released:
> > - 1 maintenance release of 2.6
> > - 3 maintenance releases of 2.7
> > - 3 maintenance releases of 2.8
> > - 3 releases of 2.9
> > - 4 releases of 3.0
> > - 2 releases of 3.1
> >
> > Total 16 releases in 2018.
> >
> > In 2019, by far we only have two releases:
> > - 1 maintenance release of 3.1
> > - 1 minor release of 3.2.
> >
> > However, the community put a lot of efforts to stabilize features of
> > various release branches.
> > There're:
> > - 217 fixed patches in 3.1.3 [1]
> > - 388 fixed patches in 3.2.1 [2]
> > - 1172 fixed patches in 3.3.0 [3] (OMG!)
> >
> > I think it is the time to do maintenance releases of 3.1/3.2 and do a
> minor
> > release for 3.3.0.
> >
> > In addition, I saw community discussion to do a 2.8.6 release for
> security
> > fixes.
> >
> > Any other releases? I think there're release plans for Ozone as well. And
> > please add your thoughts.
> >
> > Volunteers welcome! If you have interests to run a release as Release
> > Manager (or co-Resource Manager), please respond to this email thread so
> we
> > can coordinate.
> >
> > Thanks,
> > Wangda Tan
> >
> > [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution = Fixed AND
> > fixVersion = 3.1.3
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution = Fixed AND
> > fixVersion = 3.2.1
> > [3] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution = Fixed AND
> > fixVersion = 3.3.0
> >
>

Re: [ANNOUNCE] New Apache Hadoop Committer - Tao Yang

[zhankun tang]

Hi Tao,

Congratulations!!

BR,
Zhankun

On Tue, 16 Jul 2019 at 10:49, Wangda Tan  wrote:

> Congrats!
>
> Best,
> Wangda
>
> On Tue, Jul 16, 2019 at 10:37 AM 杨弢(杨弢) 
> wrote:
>
> > Thanks everyone.
> > I'm so honored to be an Apache Hadoop Committer, I will keep working on
> > this great project and contribute more. Thanks.
> >
> > Best Regards,
> > Tao Yang
> >
> >
> > --
> > 发件人：Naganarasimha Garla 
> > 发送时间：2019年7月15日(星期一) 17:55
> > 收件人：Weiwei Yang 
> > 抄 送：yarn-dev ; Hadoop Common <
> > common-dev@hadoop.apache.org>; mapreduce-dev <
> > mapreduce-...@hadoop.apache.org>; Hdfs-dev 
> > 主 题：Re: [ANNOUNCE] New Apache Hadoop Committer - Tao Yang
> >
> > Congrats and welcome Tao Yang!
> >
> > Regards
> > + Naga
> >
> > On Mon, 15 Jul 2019, 17:54 Weiwei Yang,  wrote:
> >
> > > Hi Dear Apache Hadoop Community
> > >
> > > It's my pleasure to announce that Tao Yang has been elected as an
> Apache
> > > Hadoop committer, this is to recognize his contributions to Apache
> Hadoop
> > > YARN project.
> > >
> > > Congratulations and welcome on board!
> > >
> > > Weiwei
> > > (On behalf of the Apache Hadoop PMC)
> > >
> >
> >
>