Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-23 Thread Vinod Vavilapalli 
Tx Eric, given the long cycle for 2.6.1, I’ll put MAPREDUCE-6334 into 2.6.2.

Thanks
+Vinod

On Sep 21, 2015, at 11:19 AM, Eric Payne 
> wrote:

Hi Vinod and everyone else who helped on this release!

Thank you very much for going through the work and effort to put this release 
together.

While doing my testing for Hadoop 2.6.1 RC1, I encountered the following issue:
[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException 
during InMemoryMapOutput shuffle handler - ASF 
JIRA












[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException 
during ...
We are seeing this happen when an NM's disk goes bad during the creation of map 
output(s) the reducer's fetcher can read the shuffle header and reserve the 
memory


View on issues.apache.org

Preview by Yahoo





This is actually something we ran across frequently when running 2.6. You may 
want to consider pulling that in to 2.6.

Other than that, it looks fine. I did the following manual testing on a 
one-node cluster:
- successfully downloaded and compiled the code
- successfully ran streaming jobs
- successfully ran distributed shell jobs
-- as part of the distributed shell testing, I started the jobs with the 
'-keep_containers_across_application_attempts' property set, killed the AM 
container, and verified that it would in fact keep the containers running 
across AM restarts.
- sucessfully ran wordcount jobs.

Thank you,
-Eric Payne


From: Vinod Kumar Vavilapalli >
To: common-dev@hadoop.apache.org; 
yarn-...@hadoop.apache.org; 
hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org
Cc: vino...@apache.org; Sangjin Lee 
>; Akira AJISAKA 
>
Sent: Wednesday, September 16, 2015 9:10 PM
Subject: [VOTE] Release Apache Hadoop 2.6.1 RC1

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
YARN-2890) that Jeff Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via 
repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
-  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
non-committed but desired patches. 2.6.1 is already big as is and is late
by any standard, we can definitely include them in the next release.
- The 2.6.1 wiki page [3] captures some (but not all) of the context of
the patches that we pushed in.
- Given the number of fixes pushed [4] in, we had to make a bunch of
changes to our original plan - we added a few improvements that helped us
backport patches easier (or in many cases made backports possible), and we
dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
HDFS-7611, HDFS-7843, HDFS-8850).
- I ran all the unit tests which (surprisingly?) passed. (Except for one,
which pointed out a missing fix HDFS-7552).

As discussed before [5]
- This release is the first point release after 2.6.0
- I’d like to use this as a starting release for 2.6.2 in a few weeks and
then follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread:
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets:
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
-Working-Notes
[4] List of 2.6.1 patches pushed:
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release:
http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
- Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
2.6.1.
- The additional patches in RC1 that got into 2.6.1 all the way

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-23 Thread Vinod Vavilapalli 
I’ve been out sick for a while, so couldn’t close this in time. Doing so now.

+Vinod

On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli 
> wrote:

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and 
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release 
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes for 
the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779, YARN-3248), 
additional fix to avoid incompatibility (YARN-3740), other UI bugs (YARN-1884, 
YARN-3544) and the MiniYARNCluster issue (right patch for YARN-2890) that Jeff 
Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via 
repository.apache.org at 
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
 -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] - 
non-committed but desired patches. 2.6.1 is already big as is and is late by 
any standard, we can definitely include them in the next release.
 - The 2.6.1 wiki page [3] captures some (but not all) of the context of the 
patches that we pushed in.
 - Given the number of fixes pushed [4] in, we had to make a bunch of changes 
to our original plan - we added a few improvements that helped us backport 
patches easier (or in many cases made backports possible), and we dropped a few 
that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676, HDFS-7611, HDFS-7843, 
HDFS-8850).
 - I ran all the unit tests which (surprisingly?) passed. (Except for one, 
which pointed out a missing fix HDFS-7552).

As discussed before [5]
 - This release is the first point release after 2.6.0
 - I’d like to use this as a starting release for 2.6.2 in a few weeks and then 
follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread: 
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets: 
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1-Working-Notes
[4] List of 2.6.1 patches pushed: 
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release: http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
 - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these 
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off 2.6.1.
 - The additional patches in RC1 that got into 2.6.1 all the way from 2.8 are 
NOT in 2.7.2 yet, this will be done as a followup.

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-22 Thread Masatake Iwasaki 

+1(non-binding)

- verified mds and signature of source and binary tarball
- built from source tarball with -Pnative on CentOS 7 and OpenJDK 7
- built site documentation
- started pseudo distributed cluster and ran example jobs

Masatake Iwasaki


On 9/17/15 11:10, Vinod Kumar Vavilapalli wrote:

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
YARN-2890) that Jeff Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
  -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
non-committed but desired patches. 2.6.1 is already big as is and is late
by any standard, we can definitely include them in the next release.
  - The 2.6.1 wiki page [3] captures some (but not all) of the context of
the patches that we pushed in.
  - Given the number of fixes pushed [4] in, we had to make a bunch of
changes to our original plan - we added a few improvements that helped us
backport patches easier (or in many cases made backports possible), and we
dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
HDFS-7611, HDFS-7843, HDFS-8850).
  - I ran all the unit tests which (surprisingly?) passed. (Except for one,
which pointed out a missing fix HDFS-7552).

As discussed before [5]
  - This release is the first point release after 2.6.0
  - I’d like to use this as a starting release for 2.6.2 in a few weeks and
then follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread:
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets:
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
-Working-Notes
[4] List of 2.6.1 patches pushed:
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release:
http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
  - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
2.6.1.
  - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
are NOT in 2.7.2 yet, this will be done as a followup.

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-22 Thread Karthik Kambatla 
+1 (binding)

Ran a few MR jobs on a pseudo-distributed cluster on Java 8.

On Tue, Sep 22, 2015 at 8:09 AM, Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:

> +1(non-binding)
>
> - verified mds and signature of source and binary tarball
> - built from source tarball with -Pnative on CentOS 7 and OpenJDK 7
> - built site documentation
> - started pseudo distributed cluster and ran example jobs
>
> Masatake Iwasaki
>
>
>
> On 9/17/15 11:10, Vinod Kumar Vavilapalli wrote:
>
>> Hi all,
>>
>> After a nearly month long [1] toil, with loads of help from Sangjin Lee
>> and
>> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
>> candidate RC1 for hadoop-2.6.1.
>>
>> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
>> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
>> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
>> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
>> YARN-2890) that Jeff Zhang raised.
>>
>> The RC is available at:
>> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
>>
>> The RC tag in git is: release-2.6.1-RC1
>>
>> The maven artifacts are available via repository.apache.org at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1021
>>
>> Some notes from our release process
>>   -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
>> non-committed but desired patches. 2.6.1 is already big as is and is late
>> by any standard, we can definitely include them in the next release.
>>   - The 2.6.1 wiki page [3] captures some (but not all) of the context of
>> the patches that we pushed in.
>>   - Given the number of fixes pushed [4] in, we had to make a bunch of
>> changes to our original plan - we added a few improvements that helped us
>> backport patches easier (or in many cases made backports possible), and we
>> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
>> HDFS-7611, HDFS-7843, HDFS-8850).
>>   - I ran all the unit tests which (surprisingly?) passed. (Except for
>> one,
>> which pointed out a missing fix HDFS-7552).
>>
>> As discussed before [5]
>>   - This release is the first point release after 2.6.0
>>   - I’d like to use this as a starting release for 2.6.2 in a few weeks
>> and
>> then follow up with more of these.
>>
>> Please try the release and vote; the vote will run for the usual 5 days.
>>
>> Thanks,
>> Vinod
>>
>> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
>> [1] Hadoop 2.6.1 Release process thread:
>> http://markmail.org/thread/wkbgkxkhntx5tlux
>> [2] 2.6.1 Pending tickets:
>> https://issues.apache.org/jira/issues/?filter=12331711
>> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
>> -Working-Notes
>> [4] List of 2.6.1 patches pushed:
>> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
>> %20and%20labels%20%3D%20%222.6.1-candidate%22
>> [5] Planning Hadoop 2.6.1 release:
>> http://markmail.org/thread/sbykjn5xgnksh6wg
>>
>> PS:
>>   - Note that branch-2.6 which will be the base for 2.6.2 doesn't have
>> these
>> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
>> 2.6.1.
>>   - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
>> are NOT in 2.7.2 yet, this will be done as a followup.
>>
>>
>

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Junping Du 
+1 (binding) with:
- Download and verify signatures.
- Build and deploy a single node cluster.
- Run some example jobs (PI, etc.) on deployed cluster.

Thanks,

Junping

From: Rohith Sharma K S <rohithsharm...@huawei.com>
Sent: Monday, September 21, 2015 6:53 AM
To: hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Cc: common-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: RE: [VOTE] Release Apache Hadoop 2.6.1 RC1

+1 (non-binding)

Downloaded source, built package and installed 5 node cluster.
1. Verified for RMHA/RMRestart/RMWorkpreservingRestart cluster in 
secure/non-secure mode.
2. Attached JCarder tool to cluster for identifying deadlock cycles, No cycles 
found.
3. Verified cluster for high availability.
4. Ran sample MR applications with RM HA enabled.
5. Run sanity test cases and working fine.


Thanks & Regards
Rohith Sharma K S

-Original Message-
From: sjl...@gmail.com [mailto:sjl...@gmail.com] On Behalf Of Sangjin Lee
Sent: 18 September 2015 09:50
To: yarn-...@hadoop.apache.org
Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

+1 (non-binding)

Verified the signatures, set up a pseudo-distributed cluster, ran several test 
jobs, and ran an uber job. Also verified that the UI issue I saw on
RC0 is now gone. Thanks Vinod!

Sangjin

On Thu, Sep 17, 2015 at 7:24 PM, Jian He <j...@hortonworks.com> wrote:

> +1 (binding)
>
> Build from source code.
> Deployed a local cluster.
> Validated sample jobs passed.
>
> Jian
>
> > On Sep 18, 2015, at 7:34 AM, Wangda Tan <wheele...@gmail.com> wrote:
> >
> > Deployed a local cluster, verified configured cluster with node
> > labels,
> run
> > jobs with/without node labels.
> >
> > +1 (non-binding)
> >
> > Thanks!
> >
> > On Thu, Sep 17, 2015 at 2:40 PM, Xuan Gong <xg...@hortonworks.com>
> wrote:
> >
> >> Update my vote from +1 (non-binding) to +1 binding
> >>
> >> Thanks
> >>
> >> Xuan Gong
> >>
> >>> On Sep 17, 2015, at 2:05 PM, Xuan Gong <xg...@hortonworks.com> wrote:
> >>>
> >>> +1 (non-binding)
> >>> Download and compile the source code, run several MR jobs.
> >>>
> >>> Xuan Gong
> >>>
> >>>> On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli <
> >> vino...@apache.org> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> After a nearly month long [1] toil, with loads of help from
> >>>> Sangjin
> Lee
> >> and
> >>>> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a
> >> release
> >>>> candidate RC1 for hadoop-2.6.1.
> >>>>
> >>>> RC1 is RC0 [0] (for which I opened and closed a vote last week) +
> >>>> UI
> >> fixes
> >>>> for the issue Sangjin raised (YARN-3171 and the dependencies
> YARN-3779,
> >>>> YARN-3248), additional fix to avoid incompatibility (YARN-3740),
> >>>> other
> >> UI
> >>>> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right
> >>>> patch
> >> for
> >>>> YARN-2890) that Jeff Zhang raised.
> >>>>
> >>>> The RC is available at:
> >> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
> >>>>
> >>>> The RC tag in git is: release-2.6.1-RC1
> >>>>
> >>>> The maven artifacts are available via repository.apache.org at
> >>>>
> https://repository.apache.org/content/repositories/orgapachehadoop-102
> 1
> >>>>
> >>>> Some notes from our release process
> >>>> -  - Sangjin and I moved out a bunch of items pending from 2.6.1
> >>>> [2] - non-committed but desired patches. 2.6.1 is already big as
> >>>> is and is
> >> late
> >>>> by any standard, we can definitely include them in the next release.
> >>>> - The 2.6.1 wiki page [3] captures some (but not all) of the
> >>>> context
> of
> >>>> the patches that we pushed in.
> >>>> - Given the number of fixes pushed [4] in, we had to make a bunch
> >>>> of changes to our original plan - we added a few improvements
> >>>> that helped
> >> us
> >>>> backport patches easier (or in many cases made backports
> >>>> possible),
> and
> >> we
> >>>> dropped a few that didn'

RE: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Brahma Reddy Battula 
+1 (non-binding) 
--Build from source code
--Set up a pseudo-distributed cluster and Distributed HA Cluster
--Ran Several jobs Slive,Terasort and pi.
--All are working fine.


The footer of the Web UI "Hadoop, 2014" is old. ( I think,we can include 
HDFS-8149)


Thanks & Regards
 Brahma Reddy Battula

From: Junping Du [j...@hortonworks.com]
Sent: Monday, September 21, 2015 12:12 PM
To: hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Cc: common-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

+1 (binding) with:
- Download and verify signatures.
- Build and deploy a single node cluster.
- Run some example jobs (PI, etc.) on deployed cluster.

Thanks,

Junping

From: Rohith Sharma K S <rohithsharm...@huawei.com>
Sent: Monday, September 21, 2015 6:53 AM
To: hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org
Cc: common-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: RE: [VOTE] Release Apache Hadoop 2.6.1 RC1

+1 (non-binding)

Downloaded source, built package and installed 5 node cluster.
1. Verified for RMHA/RMRestart/RMWorkpreservingRestart cluster in 
secure/non-secure mode.
2. Attached JCarder tool to cluster for identifying deadlock cycles, No cycles 
found.
3. Verified cluster for high availability.
4. Ran sample MR applications with RM HA enabled.
5. Run sanity test cases and working fine.


Thanks & Regards
Rohith Sharma K S

-Original Message-
From: sjl...@gmail.com [mailto:sjl...@gmail.com] On Behalf Of Sangjin Lee
Sent: 18 September 2015 09:50
To: yarn-...@hadoop.apache.org
Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

+1 (non-binding)

Verified the signatures, set up a pseudo-distributed cluster, ran several test 
jobs, and ran an uber job. Also verified that the UI issue I saw on
RC0 is now gone. Thanks Vinod!

Sangjin

On Thu, Sep 17, 2015 at 7:24 PM, Jian He <j...@hortonworks.com> wrote:

> +1 (binding)
>
> Build from source code.
> Deployed a local cluster.
> Validated sample jobs passed.
>
> Jian
>
> > On Sep 18, 2015, at 7:34 AM, Wangda Tan <wheele...@gmail.com> wrote:
> >
> > Deployed a local cluster, verified configured cluster with node
> > labels,
> run
> > jobs with/without node labels.
> >
> > +1 (non-binding)
> >
> > Thanks!
> >
> > On Thu, Sep 17, 2015 at 2:40 PM, Xuan Gong <xg...@hortonworks.com>
> wrote:
> >
> >> Update my vote from +1 (non-binding) to +1 binding
> >>
> >> Thanks
> >>
> >> Xuan Gong
> >>
> >>> On Sep 17, 2015, at 2:05 PM, Xuan Gong <xg...@hortonworks.com> wrote:
> >>>
> >>> +1 (non-binding)
> >>> Download and compile the source code, run several MR jobs.
> >>>
> >>> Xuan Gong
> >>>
> >>>> On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli <
> >> vino...@apache.org> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> After a nearly month long [1] toil, with loads of help from
> >>>> Sangjin
> Lee
> >> and
> >>>> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a
> >> release
> >>>> candidate RC1 for hadoop-2.6.1.
> >>>>
> >>>> RC1 is RC0 [0] (for which I opened and closed a vote last week) +
> >>>> UI
> >> fixes
> >>>> for the issue Sangjin raised (YARN-3171 and the dependencies
> YARN-3779,
> >>>> YARN-3248), additional fix to avoid incompatibility (YARN-3740),
> >>>> other
> >> UI
> >>>> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right
> >>>> patch
> >> for
> >>>> YARN-2890) that Jeff Zhang raised.
> >>>>
> >>>> The RC is available at:
> >> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
> >>>>
> >>>> The RC tag in git is: release-2.6.1-RC1
> >>>>
> >>>> The maven artifacts are available via repository.apache.org at
> >>>>
> https://repository.apache.org/content/repositories/orgapachehadoop-102
> 1
> >>>>
> >>>> Some notes from our release process
> >>>> -  - Sangjin and I moved out a bunch of items pending from 2.6.1
> >>>> [2] - non-committed but desired patches. 2.6.1 is already big as
> >>>> is and is
> >> late
> >>>> by any standard, w

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Tsuyoshi Ozawa 
+1 (non-binding)

1. Built from source.
2. Verified signature of bin tar ball and source tar ball.
3. Running some examples.
4. Running Spark on YARN with spark-shell. It works well.

I think 4 verifies compatibility is preserved.
Thanks everyone for working to release 2.6.1.

Thanks,
- Tsuyoshi

On Thu, Sep 17, 2015 at 11:10 AM, Vinod Kumar Vavilapalli
 wrote:
> Hi all,
>
> After a nearly month long [1] toil, with loads of help from Sangjin Lee and
> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
> candidate RC1 for hadoop-2.6.1.
>
> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
> YARN-2890) that Jeff Zhang raised.
>
> The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
>
> The RC tag in git is: release-2.6.1-RC1
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1021
>
> Some notes from our release process
>  -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
> non-committed but desired patches. 2.6.1 is already big as is and is late
> by any standard, we can definitely include them in the next release.
>  - The 2.6.1 wiki page [3] captures some (but not all) of the context of
> the patches that we pushed in.
>  - Given the number of fixes pushed [4] in, we had to make a bunch of
> changes to our original plan - we added a few improvements that helped us
> backport patches easier (or in many cases made backports possible), and we
> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
> HDFS-7611, HDFS-7843, HDFS-8850).
>  - I ran all the unit tests which (surprisingly?) passed. (Except for one,
> which pointed out a missing fix HDFS-7552).
>
> As discussed before [5]
>  - This release is the first point release after 2.6.0
>  - I’d like to use this as a starting release for 2.6.2 in a few weeks and
> then follow up with more of these.
>
> Please try the release and vote; the vote will run for the usual 5 days.
>
> Thanks,
> Vinod
>
> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
> [1] Hadoop 2.6.1 Release process thread:
> http://markmail.org/thread/wkbgkxkhntx5tlux
> [2] 2.6.1 Pending tickets:
> https://issues.apache.org/jira/issues/?filter=12331711
> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
> -Working-Notes
> [4] List of 2.6.1 patches pushed:
> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
> %20and%20labels%20%3D%20%222.6.1-candidate%22
> [5] Planning Hadoop 2.6.1 release:
> http://markmail.org/thread/sbykjn5xgnksh6wg
>
> PS:
>  - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
> 2.6.1.
>  - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
> are NOT in 2.7.2 yet, this will be done as a followup.

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Kuhu Shukla 
Thanks everyone! I checked the following things and did not find any issues.* 
Compilation* Running unit tests under org.apache.hadoop.yarn.server package* 
Bringing up a single node cluster* Running  a WordCount job
Regards,Kuhu Shukla

 On Monday, September 21, 2015 2:55 PM, Chang Li  
wrote:
   

 Thanks everyone who helped on this release!
Have run compilation and run various job on single node cluster. Have also
test my contribution for YARN-3267
 and verified that all
related unit tests pass.

Thank you,
Chang Li

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Chang Li 
Thanks everyone who helped on this release!
Have run compilation and run various job on single node cluster. Have also
test my contribution for YARN-3267
 and verified that all
related unit tests pass.

Thank you,
Chang Li

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Eric Payne 
Hi Vinod and everyone else who helped on this release!

Thank you very much for going through the work and effort to put this release 
together.
While doing my testing for Hadoop 2.6.1 RC1, I encountered the following 
issue:[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon 
IOException during InMemoryMapOutput shuffle handler - ASF JIRA

|   |
|   |   |   |   |   |
| [MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException 
during ...We are seeing this happen when an NM's disk goes bad during the 
creation of map output(s) the reducer's fetcher can read the shuffle header and 
reserve the memory  |
|  |
| View on issues.apache.org | Preview by Yahoo |
|  |
|   |


This is actually something we ran across frequently when running 2.6. You may 
want to consider pulling that in to 2.6.
Other than that, it looks fine. I did the following manual testing on a 
one-node cluster:
- successfully downloaded and compiled the code- successfully ran streaming 
jobs- successfully ran distributed shell jobs-- as part of the distributed 
shell testing, I started the jobs with the 
'-keep_containers_across_application_attempts' property set, killed the AM 
container, and verified that it would in fact keep the containers running 
across AM restarts.- sucessfully ran wordcount jobs.
Thank you,
-Eric Payne
 
  From: Vinod Kumar Vavilapalli 
 To: common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; 
hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org 
Cc: vino...@apache.org; Sangjin Lee ; Akira AJISAKA 
 
 Sent: Wednesday, September 16, 2015 9:10 PM
 Subject: [VOTE] Release Apache Hadoop 2.6.1 RC1
   
Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
YARN-2890) that Jeff Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
 -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
non-committed but desired patches. 2.6.1 is already big as is and is late
by any standard, we can definitely include them in the next release.
 - The 2.6.1 wiki page [3] captures some (but not all) of the context of
the patches that we pushed in.
 - Given the number of fixes pushed [4] in, we had to make a bunch of
changes to our original plan - we added a few improvements that helped us
backport patches easier (or in many cases made backports possible), and we
dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
HDFS-7611, HDFS-7843, HDFS-8850).
 - I ran all the unit tests which (surprisingly?) passed. (Except for one,
which pointed out a missing fix HDFS-7552).

As discussed before [5]
 - This release is the first point release after 2.6.0
 - I’d like to use this as a starting release for 2.6.2 in a few weeks and
then follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread:
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets:
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
-Working-Notes
[4] List of 2.6.1 patches pushed:
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release:
http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
 - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
2.6.1.
 - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
are NOT in 2.7.2 yet, this will be done as a followup.

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Akira AJISAKA 

+1 (non-binding) if MAPREDUCE-6334 is addressed.

- Downloaded and verified signatures and checksums
- Deployed a single node cluster
- Built Tez 0.7.0 with Hadoop 2.6.1 pom
- Built Hive 1.2.1 with Hadoop 2.6.1 pom
- Ran some Hive on Tez queries

Thanks everyone who worked on this release.

-Akira

On 9/22/15 03:19, Eric Payne wrote:

Hi Vinod and everyone else who helped on this release!

Thank you very much for going through the work and effort to put this release 
together.
While doing my testing for Hadoop 2.6.1 RC1, I encountered the following 
issue:[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon 
IOException during InMemoryMapOutput shuffle handler - ASF JIRA

|   |
|   |   |   |   |   |
| [MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException 
during ...We are seeing this happen when an NM's disk goes bad during the 
creation of map output(s) the reducer's fetcher can read the shuffle header and 
reserve the memory  |
|  |
| View on issues.apache.org | Preview by Yahoo |
|  |
|   |


This is actually something we ran across frequently when running 2.6. You may 
want to consider pulling that in to 2.6.
Other than that, it looks fine. I did the following manual testing on a 
one-node cluster:
- successfully downloaded and compiled the code- successfully ran streaming 
jobs- successfully ran distributed shell jobs-- as part of the distributed 
shell testing, I started the jobs with the 
'-keep_containers_across_application_attempts' property set, killed the AM 
container, and verified that it would in fact keep the containers running 
across AM restarts.- sucessfully ran wordcount jobs.
Thank you,
-Eric Payne

   From: Vinod Kumar Vavilapalli 
  To: common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org; 
hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Cc: vino...@apache.org; Sangjin Lee ; Akira AJISAKA 

  Sent: Wednesday, September 16, 2015 9:10 PM
  Subject: [VOTE] Release Apache Hadoop 2.6.1 RC1

Hi all,

After a nearly month long [1] toil, with loads of help from Sangjin Lee and
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
candidate RC1 for hadoop-2.6.1.

RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
YARN-2890) that Jeff Zhang raised.

The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/

The RC tag in git is: release-2.6.1-RC1

The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1021

Some notes from our release process
  -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
non-committed but desired patches. 2.6.1 is already big as is and is late
by any standard, we can definitely include them in the next release.
  - The 2.6.1 wiki page [3] captures some (but not all) of the context of
the patches that we pushed in.
  - Given the number of fixes pushed [4] in, we had to make a bunch of
changes to our original plan - we added a few improvements that helped us
backport patches easier (or in many cases made backports possible), and we
dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
HDFS-7611, HDFS-7843, HDFS-8850).
  - I ran all the unit tests which (surprisingly?) passed. (Except for one,
which pointed out a missing fix HDFS-7552).

As discussed before [5]
  - This release is the first point release after 2.6.0
  - I’d like to use this as a starting release for 2.6.2 in a few weeks and
then follow up with more of these.

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread:
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets:
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
-Working-Notes
[4] List of 2.6.1 patches pushed:
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release:
http://markmail.org/thread/sbykjn5xgnksh6wg

PS:
  - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
2.6.1.
  - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
are NOT in 2.7.2 yet, this will be done as a followup.

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Mit Desai 
+1 (non-binding)

# Verified signatures
# Built from source
# Deployed a single node cluster.
# Run example jobs on single-node cluster.

Thanks Vinod and all others for the effort of putting the release together.

Thanks,
Mit Desai

On Mon, Sep 21, 2015 at 1:19 PM, Eric Payne <
eric.payne1...@yahoo.com.invalid> wrote:

> Hi Vinod and everyone else who helped on this release!
>
> Thank you very much for going through the work and effort to put this
> release together.
> While doing my testing for Hadoop 2.6.1 RC1, I encountered the following
> issue:[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon
> IOException during InMemoryMapOutput shuffle handler - ASF JIRA
>
> |   |
> |   |   |   |   |   |
> | [MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon
> IOException during ...We are seeing this happen when an NM's disk goes bad
> during the creation of map output(s) the reducer's fetcher can read the
> shuffle header and reserve the memory  |
> |  |
> | View on issues.apache.org | Preview by Yahoo |
> |  |
> |   |
>
>
> This is actually something we ran across frequently when running 2.6. You
> may want to consider pulling that in to 2.6.
> Other than that, it looks fine. I did the following manual testing on a
> one-node cluster:
> - successfully downloaded and compiled the code- successfully ran
> streaming jobs- successfully ran distributed shell jobs-- as part of the
> distributed shell testing, I started the jobs with the
> '-keep_containers_across_application_attempts' property set, killed the AM
> container, and verified that it would in fact keep the containers running
> across AM restarts.- sucessfully ran wordcount jobs.
> Thank you,
> -Eric Payne
>
>   From: Vinod Kumar Vavilapalli 
>  To: common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org;
> hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
> Cc: vino...@apache.org; Sangjin Lee ; Akira AJISAKA <
> ajisa...@oss.nttdata.co.jp>
>  Sent: Wednesday, September 16, 2015 9:10 PM
>  Subject: [VOTE] Release Apache Hadoop 2.6.1 RC1
>
> Hi all,
>
> After a nearly month long [1] toil, with loads of help from Sangjin Lee and
> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
> candidate RC1 for hadoop-2.6.1.
>
> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
> YARN-2890) that Jeff Zhang raised.
>
> The RC is available at:
> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
>
> The RC tag in git is: release-2.6.1-RC1
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1021
>
> Some notes from our release process
>  -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
> non-committed but desired patches. 2.6.1 is already big as is and is late
> by any standard, we can definitely include them in the next release.
>  - The 2.6.1 wiki page [3] captures some (but not all) of the context of
> the patches that we pushed in.
>  - Given the number of fixes pushed [4] in, we had to make a bunch of
> changes to our original plan - we added a few improvements that helped us
> backport patches easier (or in many cases made backports possible), and we
> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
> HDFS-7611, HDFS-7843, HDFS-8850).
>  - I ran all the unit tests which (surprisingly?) passed. (Except for one,
> which pointed out a missing fix HDFS-7552).
>
> As discussed before [5]
>  - This release is the first point release after 2.6.0
>  - I’d like to use this as a starting release for 2.6.2 in a few weeks and
> then follow up with more of these.
>
> Please try the release and vote; the vote will run for the usual 5 days.
>
> Thanks,
> Vinod
>
> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
> [1] Hadoop 2.6.1 Release process thread:
> http://markmail.org/thread/wkbgkxkhntx5tlux
> [2] 2.6.1 Pending tickets:
> https://issues.apache.org/jira/issues/?filter=12331711
> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
> -Working-Notes
> [4] List of 2.6.1 patches pushed:
> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
> %20and%20labels%20%3D%20%222.6.1-candidate%22
> [5] Planning Hadoop 2.6.1 release:
> http://markmail.org/thread/sbykjn5xgnksh6wg
>
> PS:
>  - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
> 2.6.1.
>  - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
> are NOT in 2.7.2 yet, this will be done as a followup.
>
>
>

RE: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-20 Thread Rohith Sharma K S 
+1 (non-binding)

Downloaded source, built package and installed 5 node cluster.
1. Verified for RMHA/RMRestart/RMWorkpreservingRestart cluster in 
secure/non-secure mode.
2. Attached JCarder tool to cluster for identifying deadlock cycles, No cycles 
found.
3. Verified cluster for high availability.
4. Ran sample MR applications with RM HA enabled. 
5. Run sanity test cases and working fine.


Thanks & Regards
Rohith Sharma K S

-Original Message-
From: sjl...@gmail.com [mailto:sjl...@gmail.com] On Behalf Of Sangjin Lee
Sent: 18 September 2015 09:50
To: yarn-...@hadoop.apache.org
Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

+1 (non-binding)

Verified the signatures, set up a pseudo-distributed cluster, ran several test 
jobs, and ran an uber job. Also verified that the UI issue I saw on
RC0 is now gone. Thanks Vinod!

Sangjin

On Thu, Sep 17, 2015 at 7:24 PM, Jian He <j...@hortonworks.com> wrote:

> +1 (binding)
>
> Build from source code.
> Deployed a local cluster.
> Validated sample jobs passed.
>
> Jian
>
> > On Sep 18, 2015, at 7:34 AM, Wangda Tan <wheele...@gmail.com> wrote:
> >
> > Deployed a local cluster, verified configured cluster with node 
> > labels,
> run
> > jobs with/without node labels.
> >
> > +1 (non-binding)
> >
> > Thanks!
> >
> > On Thu, Sep 17, 2015 at 2:40 PM, Xuan Gong <xg...@hortonworks.com>
> wrote:
> >
> >> Update my vote from +1 (non-binding) to +1 binding
> >>
> >> Thanks
> >>
> >> Xuan Gong
> >>
> >>> On Sep 17, 2015, at 2:05 PM, Xuan Gong <xg...@hortonworks.com> wrote:
> >>>
> >>> +1 (non-binding)
> >>> Download and compile the source code, run several MR jobs.
> >>>
> >>> Xuan Gong
> >>>
> >>>> On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli <
> >> vino...@apache.org> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> After a nearly month long [1] toil, with loads of help from 
> >>>> Sangjin
> Lee
> >> and
> >>>> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a
> >> release
> >>>> candidate RC1 for hadoop-2.6.1.
> >>>>
> >>>> RC1 is RC0 [0] (for which I opened and closed a vote last week) + 
> >>>> UI
> >> fixes
> >>>> for the issue Sangjin raised (YARN-3171 and the dependencies
> YARN-3779,
> >>>> YARN-3248), additional fix to avoid incompatibility (YARN-3740), 
> >>>> other
> >> UI
> >>>> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right 
> >>>> patch
> >> for
> >>>> YARN-2890) that Jeff Zhang raised.
> >>>>
> >>>> The RC is available at:
> >> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
> >>>>
> >>>> The RC tag in git is: release-2.6.1-RC1
> >>>>
> >>>> The maven artifacts are available via repository.apache.org at
> >>>>
> https://repository.apache.org/content/repositories/orgapachehadoop-102
> 1
> >>>>
> >>>> Some notes from our release process
> >>>> -  - Sangjin and I moved out a bunch of items pending from 2.6.1 
> >>>> [2] - non-committed but desired patches. 2.6.1 is already big as 
> >>>> is and is
> >> late
> >>>> by any standard, we can definitely include them in the next release.
> >>>> - The 2.6.1 wiki page [3] captures some (but not all) of the 
> >>>> context
> of
> >>>> the patches that we pushed in.
> >>>> - Given the number of fixes pushed [4] in, we had to make a bunch 
> >>>> of changes to our original plan - we added a few improvements 
> >>>> that helped
> >> us
> >>>> backport patches easier (or in many cases made backports 
> >>>> possible),
> and
> >> we
> >>>> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, 
> >>>> HDFS-7676, HDFS-7611, HDFS-7843, HDFS-8850).
> >>>> - I ran all the unit tests which (surprisingly?) passed. (Except 
> >>>> for
> >> one,
> >>>> which pointed out a missing fix HDFS-7552).
> >>>>
> >>>> As discussed before [5]
> >>>> - This release is the first point release after 2.6.0
> >>>

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-17 Thread Ted Yu 
Ran hbase test suite based on 2.6.1 RC1 which passed.

FYI

On Wed, Sep 16, 2015 at 7:10 PM, Vinod Kumar Vavilapalli  wrote:

> Hi all,
>
> After a nearly month long [1] toil, with loads of help from Sangjin Lee and
> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
> candidate RC1 for hadoop-2.6.1.
>
> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
> YARN-2890) that Jeff Zhang raised.
>
> The RC is available at:
> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
>
> The RC tag in git is: release-2.6.1-RC1
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1021
>
> Some notes from our release process
>  -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
> non-committed but desired patches. 2.6.1 is already big as is and is late
> by any standard, we can definitely include them in the next release.
>  - The 2.6.1 wiki page [3] captures some (but not all) of the context of
> the patches that we pushed in.
>  - Given the number of fixes pushed [4] in, we had to make a bunch of
> changes to our original plan - we added a few improvements that helped us
> backport patches easier (or in many cases made backports possible), and we
> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
> HDFS-7611, HDFS-7843, HDFS-8850).
>  - I ran all the unit tests which (surprisingly?) passed. (Except for one,
> which pointed out a missing fix HDFS-7552).
>
> As discussed before [5]
>  - This release is the first point release after 2.6.0
>  - I’d like to use this as a starting release for 2.6.2 in a few weeks and
> then follow up with more of these.
>
> Please try the release and vote; the vote will run for the usual 5 days.
>
> Thanks,
> Vinod
>
> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
> [1] Hadoop 2.6.1 Release process thread:
> http://markmail.org/thread/wkbgkxkhntx5tlux
> [2] 2.6.1 Pending tickets:
> https://issues.apache.org/jira/issues/?filter=12331711
> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
> -Working-Notes
> [4] List of 2.6.1 patches pushed:
> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
> %20and%20labels%20%3D%20%222.6.1-candidate%22
> [5] Planning Hadoop 2.6.1 release:
> http://markmail.org/thread/sbykjn5xgnksh6wg
>
> PS:
>  - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
> 2.6.1.
>  - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
> are NOT in 2.7.2 yet, this will be done as a followup.
>

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-17 Thread Xuan Gong 
+1 (non-binding)
Download and compile the source code, run several MR jobs.

Xuan Gong

> On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> Hi all,
> 
> After a nearly month long [1] toil, with loads of help from Sangjin Lee and
> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
> candidate RC1 for hadoop-2.6.1.
> 
> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
> YARN-2890) that Jeff Zhang raised.
> 
> The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
> 
> The RC tag in git is: release-2.6.1-RC1
> 
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1021
> 
> Some notes from our release process
> -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
> non-committed but desired patches. 2.6.1 is already big as is and is late
> by any standard, we can definitely include them in the next release.
> - The 2.6.1 wiki page [3] captures some (but not all) of the context of
> the patches that we pushed in.
> - Given the number of fixes pushed [4] in, we had to make a bunch of
> changes to our original plan - we added a few improvements that helped us
> backport patches easier (or in many cases made backports possible), and we
> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
> HDFS-7611, HDFS-7843, HDFS-8850).
> - I ran all the unit tests which (surprisingly?) passed. (Except for one,
> which pointed out a missing fix HDFS-7552).
> 
> As discussed before [5]
> - This release is the first point release after 2.6.0
> - I’d like to use this as a starting release for 2.6.2 in a few weeks and
> then follow up with more of these.
> 
> Please try the release and vote; the vote will run for the usual 5 days.
> 
> Thanks,
> Vinod
> 
> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
> [1] Hadoop 2.6.1 Release process thread:
> http://markmail.org/thread/wkbgkxkhntx5tlux
> [2] 2.6.1 Pending tickets:
> https://issues.apache.org/jira/issues/?filter=12331711
> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
> -Working-Notes
> [4] List of 2.6.1 patches pushed:
> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
> %20and%20labels%20%3D%20%222.6.1-candidate%22
> [5] Planning Hadoop 2.6.1 release:
> http://markmail.org/thread/sbykjn5xgnksh6wg
> 
> PS:
> - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
> 2.6.1.
> - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
> are NOT in 2.7.2 yet, this will be done as a followup.

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-17 Thread Xuan Gong 
Update my vote from +1 (non-binding) to +1 binding

Thanks

Xuan Gong

> On Sep 17, 2015, at 2:05 PM, Xuan Gong  wrote:
> 
> +1 (non-binding)
> Download and compile the source code, run several MR jobs.
> 
> Xuan Gong
> 
>> On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli  
>> wrote:
>> 
>> Hi all,
>> 
>> After a nearly month long [1] toil, with loads of help from Sangjin Lee and
>> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
>> candidate RC1 for hadoop-2.6.1.
>> 
>> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
>> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
>> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
>> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
>> YARN-2890) that Jeff Zhang raised.
>> 
>> The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
>> 
>> The RC tag in git is: release-2.6.1-RC1
>> 
>> The maven artifacts are available via repository.apache.org at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1021
>> 
>> Some notes from our release process
>> -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
>> non-committed but desired patches. 2.6.1 is already big as is and is late
>> by any standard, we can definitely include them in the next release.
>> - The 2.6.1 wiki page [3] captures some (but not all) of the context of
>> the patches that we pushed in.
>> - Given the number of fixes pushed [4] in, we had to make a bunch of
>> changes to our original plan - we added a few improvements that helped us
>> backport patches easier (or in many cases made backports possible), and we
>> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
>> HDFS-7611, HDFS-7843, HDFS-8850).
>> - I ran all the unit tests which (surprisingly?) passed. (Except for one,
>> which pointed out a missing fix HDFS-7552).
>> 
>> As discussed before [5]
>> - This release is the first point release after 2.6.0
>> - I’d like to use this as a starting release for 2.6.2 in a few weeks and
>> then follow up with more of these.
>> 
>> Please try the release and vote; the vote will run for the usual 5 days.
>> 
>> Thanks,
>> Vinod
>> 
>> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
>> [1] Hadoop 2.6.1 Release process thread:
>> http://markmail.org/thread/wkbgkxkhntx5tlux
>> [2] 2.6.1 Pending tickets:
>> https://issues.apache.org/jira/issues/?filter=12331711
>> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
>> -Working-Notes
>> [4] List of 2.6.1 patches pushed:
>> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
>> %20and%20labels%20%3D%20%222.6.1-candidate%22
>> [5] Planning Hadoop 2.6.1 release:
>> http://markmail.org/thread/sbykjn5xgnksh6wg
>> 
>> PS:
>> - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
>> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
>> 2.6.1.
>> - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
>> are NOT in 2.7.2 yet, this will be done as a followup.
>

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-17 Thread Wangda Tan 
Deployed a local cluster, verified configured cluster with node labels, run
jobs with/without node labels.

+1 (non-binding)

Thanks!

On Thu, Sep 17, 2015 at 2:40 PM, Xuan Gong  wrote:

> Update my vote from +1 (non-binding) to +1 binding
>
> Thanks
>
> Xuan Gong
>
> > On Sep 17, 2015, at 2:05 PM, Xuan Gong  wrote:
> >
> > +1 (non-binding)
> > Download and compile the source code, run several MR jobs.
> >
> > Xuan Gong
> >
> >> On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org> wrote:
> >>
> >> Hi all,
> >>
> >> After a nearly month long [1] toil, with loads of help from Sangjin Lee
> and
> >> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a
> release
> >> candidate RC1 for hadoop-2.6.1.
> >>
> >> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI
> fixes
> >> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
> >> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other
> UI
> >> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch
> for
> >> YARN-2890) that Jeff Zhang raised.
> >>
> >> The RC is available at:
> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
> >>
> >> The RC tag in git is: release-2.6.1-RC1
> >>
> >> The maven artifacts are available via repository.apache.org at
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1021
> >>
> >> Some notes from our release process
> >> -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
> >> non-committed but desired patches. 2.6.1 is already big as is and is
> late
> >> by any standard, we can definitely include them in the next release.
> >> - The 2.6.1 wiki page [3] captures some (but not all) of the context of
> >> the patches that we pushed in.
> >> - Given the number of fixes pushed [4] in, we had to make a bunch of
> >> changes to our original plan - we added a few improvements that helped
> us
> >> backport patches easier (or in many cases made backports possible), and
> we
> >> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
> >> HDFS-7611, HDFS-7843, HDFS-8850).
> >> - I ran all the unit tests which (surprisingly?) passed. (Except for
> one,
> >> which pointed out a missing fix HDFS-7552).
> >>
> >> As discussed before [5]
> >> - This release is the first point release after 2.6.0
> >> - I’d like to use this as a starting release for 2.6.2 in a few weeks
> and
> >> then follow up with more of these.
> >>
> >> Please try the release and vote; the vote will run for the usual 5 days.
> >>
> >> Thanks,
> >> Vinod
> >>
> >> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
> >> [1] Hadoop 2.6.1 Release process thread:
> >> http://markmail.org/thread/wkbgkxkhntx5tlux
> >> [2] 2.6.1 Pending tickets:
> >> https://issues.apache.org/jira/issues/?filter=12331711
> >> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
> >> -Working-Notes
> >> [4] List of 2.6.1 patches pushed:
> >> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
> >> %20and%20labels%20%3D%20%222.6.1-candidate%22
> >> [5] Planning Hadoop 2.6.1 release:
> >> http://markmail.org/thread/sbykjn5xgnksh6wg
> >>
> >> PS:
> >> - Note that branch-2.6 which will be the base for 2.6.2 doesn't have
> these
> >> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based
> off
> >> 2.6.1.
> >> - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
> >> are NOT in 2.7.2 yet, this will be done as a followup.
> >
>
>

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-17 Thread Sangjin Lee 
+1 (non-binding)

Verified the signatures, set up a pseudo-distributed cluster, ran several
test jobs, and ran an uber job. Also verified that the UI issue I saw on
RC0 is now gone. Thanks Vinod!

Sangjin

On Thu, Sep 17, 2015 at 7:24 PM, Jian He  wrote:

> +1 (binding)
>
> Build from source code.
> Deployed a local cluster.
> Validated sample jobs passed.
>
> Jian
>
> > On Sep 18, 2015, at 7:34 AM, Wangda Tan  wrote:
> >
> > Deployed a local cluster, verified configured cluster with node labels,
> run
> > jobs with/without node labels.
> >
> > +1 (non-binding)
> >
> > Thanks!
> >
> > On Thu, Sep 17, 2015 at 2:40 PM, Xuan Gong 
> wrote:
> >
> >> Update my vote from +1 (non-binding) to +1 binding
> >>
> >> Thanks
> >>
> >> Xuan Gong
> >>
> >>> On Sep 17, 2015, at 2:05 PM, Xuan Gong  wrote:
> >>>
> >>> +1 (non-binding)
> >>> Download and compile the source code, run several MR jobs.
> >>>
> >>> Xuan Gong
> >>>
>  On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli <
> >> vino...@apache.org> wrote:
> 
>  Hi all,
> 
>  After a nearly month long [1] toil, with loads of help from Sangjin
> Lee
> >> and
>  Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a
> >> release
>  candidate RC1 for hadoop-2.6.1.
> 
>  RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI
> >> fixes
>  for the issue Sangjin raised (YARN-3171 and the dependencies
> YARN-3779,
>  YARN-3248), additional fix to avoid incompatibility (YARN-3740), other
> >> UI
>  bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch
> >> for
>  YARN-2890) that Jeff Zhang raised.
> 
>  The RC is available at:
> >> http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
> 
>  The RC tag in git is: release-2.6.1-RC1
> 
>  The maven artifacts are available via repository.apache.org at
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1021
> 
>  Some notes from our release process
>  -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
>  non-committed but desired patches. 2.6.1 is already big as is and is
> >> late
>  by any standard, we can definitely include them in the next release.
>  - The 2.6.1 wiki page [3] captures some (but not all) of the context
> of
>  the patches that we pushed in.
>  - Given the number of fixes pushed [4] in, we had to make a bunch of
>  changes to our original plan - we added a few improvements that helped
> >> us
>  backport patches easier (or in many cases made backports possible),
> and
> >> we
>  dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
>  HDFS-7611, HDFS-7843, HDFS-8850).
>  - I ran all the unit tests which (surprisingly?) passed. (Except for
> >> one,
>  which pointed out a missing fix HDFS-7552).
> 
>  As discussed before [5]
>  - This release is the first point release after 2.6.0
>  - I’d like to use this as a starting release for 2.6.2 in a few weeks
> >> and
>  then follow up with more of these.
> 
>  Please try the release and vote; the vote will run for the usual 5
> days.
> 
>  Thanks,
>  Vinod
> 
>  [0] Hadoop 2.6.1 RC0 vote:
> http://markmail.org/thread/ubut2rn3lodc55iy
>  [1] Hadoop 2.6.1 Release process thread:
>  http://markmail.org/thread/wkbgkxkhntx5tlux
>  [2] 2.6.1 Pending tickets:
>  https://issues.apache.org/jira/issues/?filter=12331711
>  [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
>  -Working-Notes
>  [4] List of 2.6.1 patches pushed:
>  https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
>  %20and%20labels%20%3D%20%222.6.1-candidate%22
>  [5] Planning Hadoop 2.6.1 release:
>  http://markmail.org/thread/sbykjn5xgnksh6wg
> 
>  PS:
>  - Note that branch-2.6 which will be the base for 2.6.2 doesn't have
> >> these
>  fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based
> >> off
>  2.6.1.
>  - The additional patches in RC1 that got into 2.6.1 all the way from
> 2.8
>  are NOT in 2.7.2 yet, this will be done as a followup.
> >>>
> >>
> >>
>
>