Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

[Akira Ajisaka]

Thanks Vinod and Andrew for the summary.

> Here's an attempt at encoding this policy as a set of rules for 
setting fix

> versions (credit to Allen):
>
> 1) Set the fix version for all a.b.c versions, where c > 0.
> 2) For each major release line, set the lowest a.b.0 version.

Assuming 3.0.0-alpha1 will be released between 2.7.0 and 2.8.0, we need 
to add 3.0.0-alphaX if 2.8.0 is in the fix versions of a jira and we 
don't need to add 3.0.0-alphaX if 2.7.0 is in the fix versions of a 
jira. Is it right?


> As an example, if a JIRA was committed to branch-2.6, branch-2.7, 
branch-2,
> branch-3.0.0-alpha1, and trunk, it could have fix versions of 2.6.5, 
2.7.3,

> 2.8.0, 3.0.0-alpha1.

(nit) 2.7.3 -> 2.7.4, because branch-2.7.3 has been cut.

Regards,
Akira

On 7/27/16 06:03, Andrew Wang wrote:

Thanks Vinod for forking the thread. Let me try and summarize what Allen
and I talked about in the previous thread.

Currently, we've been marking JIRAs with fix versions of both 2.6.x and
2.7.x. IIUC, the chronological ordering between these two lines is actually
not important. If you're on 2.6.1, you can see the new changes in 2.6.2 by
looking at what has the 2.6.2 fix version, and looking at the 2.6.2 CHANGES
entries. This makes sense from a user POV.

The question now is what we do for the 2.8.0 and 3.0.0-alpha1 fix versions.
Allen's historical perspective is that we've based each minor or major
release off of the previous minor release. So, 2.8.0 would be based off of
2.7.0. Assuming 3.0.0-alpha1 happens before 2.8.0, 3.0.0-alpha1 would also
be based off of 2.7.0. This also makes sense from a user POV; someone on a
2.6.x going to 3.0.0-alpha1 can look at the 2.7.0 and 3.0.0-alpha1 notes to
see what's changed.

Having looked at the changelogs of other enterprise software products, this
is pretty standard.

Perhaps restating the obvious, but:

* For a.b.c releases, the "c"s need to be released in order.
* For a.b.0 releases, the "b"s need to be released in order.
* For a.0.0 releases, it comes after a specific x.y.0 release.

Here's an attempt at encoding this policy as a set of rules for setting fix
versions (credit to Allen):

1) Set the fix version for all a.b.c versions, where c > 0.
2) For each major release line, set the lowest a.b.0 version.

The -alphaX versions we're using leading up to 3.0.0 GA can be treated as
a.b.c versions, with alpha1 being the a.b.0 release.

As an example, if a JIRA was committed to branch-2.6, branch-2.7, branch-2,
branch-3.0.0-alpha1, and trunk, it could have fix versions of 2.6.5, 2.7.3,
2.8.0, 3.0.0-alpha1. The first two fix versions come from application of
rule 1, and the last two fix versions come from rule 2.

I'm very eager to move this discussion forward, so feel free to reach out
on or off list if I can help with anything.

Best,
Andrew




-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Akira Ajisaka]

+1 for the source tarball.

- Downloaded source tarball and binary tarball
- Verified signatures and checksums
- Compiled and built a single node cluster
- Compiled Hive 2.1.0/1.2.1 and Tez 0.8.4/0.7.1 using Hadoop 2.7.3 pom 
successfully

- Ran some Hive on Tez queries successfully

Thanks,
Akira

On 7/27/16 04:12, Vinod Kumar Vavilapalli wrote:

But, everyone please do continue your sanity checking on RC0 in case there are 
more issues to be fixed.

Thanks
+Vinod


On Jul 26, 2016, at 12:11 PM, Vinod Kumar Vavilapalli  
wrote:

Thanks Daniel and Wei.

I think these are worth fixing, I’m withdrawing this RC. Will look at fixing 
these issues and roll a new candidate with the fixes as soon as possible.

Thanks
+Vinod


On Jul 26, 2016, at 11:05 AM, Wei-Chiu Chuang > wrote:

I noticed two issues:

(1) I ran hadoop checknative, but it seems the binary tarball was not compiled 
with native library for Linux. On the contrary, the Hadoop built from source 
tarball with maven -Pnative can find the native libraries on the same host.

(2) I noticed that the release dates in CHANGES.txt in tag release-2.7.3-RC0 
are set to Release 2.7.3 - 2016-07-27.
However, the release dates in CHANGES.txt in the source and binary tar balls 
are set to Release 2.7.3 - 2016-08-01. This is probably a non-issue though.

* Downloaded source and binary.
* Verified signature.
* Verified checksum.
* Built from source using 64-bit Java 7 (1.7.0.75) and 8 (1.8.0.05). Both went 
fine.
* Ran hadoop checknative

On Tue, Jul 26, 2016 at 9:12 AM, Rushabh Shah > wrote:
Thanks Vinod for all the release work !
+1 (non-binding).
* Downloaded from source and built it.* Deployed a pseudo distributed cluster.
* Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything works 
fine.


On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli > wrote:


 Hi all,

I've created a release candidate RC0 for Apache Hadoop 2.7.3.

As discussed before, this is the next maintenance release to follow up 2.7.2.

The RC is available for validation at: http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ 
 
>

The RC tag in git is: release-2.7.3-RC0

The maven artifacts are available via repository.apache.org  
> at 
https://repository.apache.org/content/repositories/orgapachehadoop-1040/ 
 
>

The release-notes are inside the tar-balls at location 
hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I hosted this at 
http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html 
 
> for your quick 
perusal.

As you may have noted, a very long fix-cycle for the License & Notice issues 
(HADOOP-12893) caused 2.7.3 (along with every other Hadoop release) to slip by 
quite a bit. This release's related discussion thread is linked below: [1].

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[1]: 2.7.3 release plan: https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html 
 
>











-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Eric Payne]

+1 (non-binding)
Thanks, Vinod, for all of your hard work and congratulations in completing this 
release.
After downloading and building the source, I installed Hadoop 2.7.3 RC0 on a 
3-node, multi-tenant, insecure cluster. I ran manual tests to ensure the 
following:
- Ensure that user limit percent is honored for multiple users in the same queue
- Ensure that cross-queue preemption occurs when a preemptable queue is over 
its guaranteed capacity
- Ensure that jobs submitted to labelled queues only run on the labelled nodes.
- Ensure that RM UI shows correct queue resource metrics when jobs are running 
in labelled queues (YARN-4751).
- Ensure that a yarn distributed shell application can be launched and complete 
successfully.

Eric Payne
  From: Vinod Kumar Vavilapalli 
 To: "common-...@hadoop.apache.org" ; 
hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org; 
"mapreduce-dev@hadoop.apache.org"  
Cc: Vinod Kumar Vavilapalli 
 Sent: Friday, July 22, 2016 9:15 PM
 Subject: [VOTE] Release Apache Hadoop 2.7.3 RC0
   
Hi all,

I've created a release candidate RC0 for Apache Hadoop 2.7.3.

As discussed before, this is the next maintenance release to follow up 2.7.2.

The RC is available for validation at: 
http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ 


The RC tag in git is: release-2.7.3-RC0

The maven artifacts are available via repository.apache.org 
 at 
https://repository.apache.org/content/repositories/orgapachehadoop-1040/ 


The release-notes are inside the tar-balls at location 
hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I hosted 
this at http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html 
 for your 
quick perusal.

As you may have noted, a very long fix-cycle for the License & Notice issues 
(HADOOP-12893) caused 2.7.3 (along with every other Hadoop release) to slip by 
quite a bit. This release's related discussion thread is linked below: [1].

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[1]: 2.7.3 release plan: 
https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html 


   

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Andrew Wang]

On Tue, Jul 26, 2016 at 1:23 PM, Karthik Kambatla 
wrote:

> IIRR, the vote is on source artifacts and binaries are for convenience.
>
> Let me refine this statement a bit. Both the binary tarball and the JARs
we publish to Maven are still official release artifacts. This is why we
need L for them. Else, we shouldn't be distributing them.

In any case, it's up to the PMC to decide what is a valid release, via lazy
majority voting on this thread. I personally don't feel that strongly about
the native code, since I think it needs to be compiled per-distro anyway.

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Eric Badger]

* Verified mds and pgp signatures of both source and binary* Built tarball from 
source on OS X 10.11.6 (El Capitan)* Deployed in pseudo-distributed mode* Ran 
sleep jobs and other randomly selected tests on both MapReduce and Tez* 
Visually verified the RM and history server UIs 
Thanks,
Eric 

On Tuesday, July 26, 2016 3:23 PM, Karthik Kambatla  
wrote:
 

 IIRR, the vote is on source artifacts and binaries are for convenience.

If that is right, I am open to either options - do another RC or continue
this vote and fix the binary artifacts.

On Tue, Jul 26, 2016 at 12:11 PM, Vinod Kumar Vavilapalli <
vino...@apache.org> wrote:

> Thanks Daniel and Wei.
>
> I think these are worth fixing, I’m withdrawing this RC. Will look at
> fixing these issues and roll a new candidate with the fixes as soon as
> possible.
>
> Thanks
> +Vinod
>
> > On Jul 26, 2016, at 11:05 AM, Wei-Chiu Chuang 
> wrote:
> >
> > I noticed two issues:
> >
> > (1) I ran hadoop checknative, but it seems the binary tarball was not
> compiled with native library for Linux. On the contrary, the Hadoop built
> from source tarball with maven -Pnative can find the native libraries on
> the same host.
> >
> > (2) I noticed that the release dates in CHANGES.txt in tag
> release-2.7.3-RC0 are set to Release 2.7.3 - 2016-07-27.
> > However, the release dates in CHANGES.txt in the source and binary tar
> balls are set to Release 2.7.3 - 2016-08-01. This is probably a non-issue
> though.
> >
> > * Downloaded source and binary.
> > * Verified signature.
> > * Verified checksum.
> > * Built from source using 64-bit Java 7 (1.7.0.75) and 8 (1.8.0.05).
> Both went fine.
> > * Ran hadoop checknative
> >
> > On Tue, Jul 26, 2016 at 9:12 AM, Rushabh Shah
> >
> wrote:
> > Thanks Vinod for all the release work !
> > +1 (non-binding).
> > * Downloaded from source and built it.* Deployed a pseudo distributed
> cluster.
> > * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything
> works fine.
> >
> >
> >    On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org > wrote:
> >
> >
> >  Hi all,
> >
> > I've created a release candidate RC0 for Apache Hadoop 2.7.3.
> >
> > As discussed before, this is the next maintenance release to follow up
> 2.7.2.
> >
> > The RC is available for validation at:
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/> <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>>
> >
> > The RC tag in git is: release-2.7.3-RC0
> >
> > The maven artifacts are available via repository.apache.org <
> http://repository.apache.org/>  http://repository.apache.org/>> at
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/ <
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/>
>   >>
> >
> > The release-notes are inside the tar-balls at location
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I
> hosted this at
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html> <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html>>
> for your quick perusal.
> >
> > As you may have noted, a very long fix-cycle for the License & Notice
> issues (HADOOP-12893) caused 2.7.3 (along with every other Hadoop release)
> to slip by quite a bit. This release's related discussion thread is linked
> below: [1].
> >
> > Please try the release and vote; the vote will run for the usual 5 days.
> >
> > Thanks,
> > Vinod
> >
> > [1]: 2.7.3 release plan:
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html <
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html> <
> http://markmail.org/thread/6yv2fyrs4jlepmmr <
> http://markmail.org/thread/6yv2fyrs4jlepmmr>>
> >
> >
> >
>
>

  

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Karthik Kambatla]

IIRR, the vote is on source artifacts and binaries are for convenience.

If that is right, I am open to either options - do another RC or continue
this vote and fix the binary artifacts.

On Tue, Jul 26, 2016 at 12:11 PM, Vinod Kumar Vavilapalli <
vino...@apache.org> wrote:

> Thanks Daniel and Wei.
>
> I think these are worth fixing, I’m withdrawing this RC. Will look at
> fixing these issues and roll a new candidate with the fixes as soon as
> possible.
>
> Thanks
> +Vinod
>
> > On Jul 26, 2016, at 11:05 AM, Wei-Chiu Chuang 
> wrote:
> >
> > I noticed two issues:
> >
> > (1) I ran hadoop checknative, but it seems the binary tarball was not
> compiled with native library for Linux. On the contrary, the Hadoop built
> from source tarball with maven -Pnative can find the native libraries on
> the same host.
> >
> > (2) I noticed that the release dates in CHANGES.txt in tag
> release-2.7.3-RC0 are set to Release 2.7.3 - 2016-07-27.
> > However, the release dates in CHANGES.txt in the source and binary tar
> balls are set to Release 2.7.3 - 2016-08-01. This is probably a non-issue
> though.
> >
> > * Downloaded source and binary.
> > * Verified signature.
> > * Verified checksum.
> > * Built from source using 64-bit Java 7 (1.7.0.75) and 8 (1.8.0.05).
> Both went fine.
> > * Ran hadoop checknative
> >
> > On Tue, Jul 26, 2016 at 9:12 AM, Rushabh Shah
> >
> wrote:
> > Thanks Vinod for all the release work !
> > +1 (non-binding).
> > * Downloaded from source and built it.* Deployed a pseudo distributed
> cluster.
> > * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything
> works fine.
> >
> >
> > On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org > wrote:
> >
> >
> >  Hi all,
> >
> > I've created a release candidate RC0 for Apache Hadoop 2.7.3.
> >
> > As discussed before, this is the next maintenance release to follow up
> 2.7.2.
> >
> > The RC is available for validation at:
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/> <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>>
> >
> > The RC tag in git is: release-2.7.3-RC0
> >
> > The maven artifacts are available via repository.apache.org <
> http://repository.apache.org/>  http://repository.apache.org/>> at
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/ <
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/>
>   >>
> >
> > The release-notes are inside the tar-balls at location
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I
> hosted this at
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html> <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html>>
> for your quick perusal.
> >
> > As you may have noted, a very long fix-cycle for the License & Notice
> issues (HADOOP-12893) caused 2.7.3 (along with every other Hadoop release)
> to slip by quite a bit. This release's related discussion thread is linked
> below: [1].
> >
> > Please try the release and vote; the vote will run for the usual 5 days.
> >
> > Thanks,
> > Vinod
> >
> > [1]: 2.7.3 release plan:
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html <
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html> <
> http://markmail.org/thread/6yv2fyrs4jlepmmr <
> http://markmail.org/thread/6yv2fyrs4jlepmmr>>
> >
> >
> >
>
>

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Naganarasimha Garla]

Tested as follows
- deployed a pseudo cluster from RC0 tar
- Verified signature and checksum
- Run MR sleep on YARN
- With/Without node labels enabled.
- Verified ATS V1 (One small issue though not regression, tracking URL for
running app is shown as Unassigned, will check and raise if required )

Regards,
+ Naga

On Wed, Jul 27, 2016 at 12:42 AM, Vinod Kumar Vavilapalli <
vino...@apache.org> wrote:

> But, everyone please do continue your sanity checking on RC0 in case there
> are more issues to be fixed.
>
> Thanks
> +Vinod
>
> > On Jul 26, 2016, at 12:11 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org> wrote:
> >
> > Thanks Daniel and Wei.
> >
> > I think these are worth fixing, I’m withdrawing this RC. Will look at
> fixing these issues and roll a new candidate with the fixes as soon as
> possible.
> >
> > Thanks
> > +Vinod
> >
> >> On Jul 26, 2016, at 11:05 AM, Wei-Chiu Chuang  > wrote:
> >>
> >> I noticed two issues:
> >>
> >> (1) I ran hadoop checknative, but it seems the binary tarball was not
> compiled with native library for Linux. On the contrary, the Hadoop built
> from source tarball with maven -Pnative can find the native libraries on
> the same host.
> >>
> >> (2) I noticed that the release dates in CHANGES.txt in tag
> release-2.7.3-RC0 are set to Release 2.7.3 - 2016-07-27.
> >> However, the release dates in CHANGES.txt in the source and binary tar
> balls are set to Release 2.7.3 - 2016-08-01. This is probably a non-issue
> though.
> >>
> >> * Downloaded source and binary.
> >> * Verified signature.
> >> * Verified checksum.
> >> * Built from source using 64-bit Java 7 (1.7.0.75) and 8 (1.8.0.05).
> Both went fine.
> >> * Ran hadoop checknative
> >>
> >> On Tue, Jul 26, 2016 at 9:12 AM, Rushabh Shah
> >
> wrote:
> >> Thanks Vinod for all the release work !
> >> +1 (non-binding).
> >> * Downloaded from source and built it.* Deployed a pseudo distributed
> cluster.
> >> * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything
> works fine.
> >>
> >>
> >> On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org > wrote:
> >>
> >>
> >>  Hi all,
> >>
> >> I've created a release candidate RC0 for Apache Hadoop 2.7.3.
> >>
> >> As discussed before, this is the next maintenance release to follow up
> 2.7.2.
> >>
> >> The RC is available for validation at:
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/> <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>>
> >>
> >> The RC tag in git is: release-2.7.3-RC0
> >>
> >> The maven artifacts are available via repository.apache.org <
> http://repository.apache.org/>  http://repository.apache.org/>> at
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/ <
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/>
>   >>
> >>
> >> The release-notes are inside the tar-balls at location
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I
> hosted this at
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html> <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html>>
> for your quick perusal.
> >>
> >> As you may have noted, a very long fix-cycle for the License & Notice
> issues (HADOOP-12893) caused 2.7.3 (along with every other Hadoop release)
> to slip by quite a bit. This release's related discussion thread is linked
> below: [1].
> >>
> >> Please try the release and vote; the vote will run for the usual 5 days.
> >>
> >> Thanks,
> >> Vinod
> >>
> >> [1]: 2.7.3 release plan:
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html <
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html> <
> http://markmail.org/thread/6yv2fyrs4jlepmmr <
> http://markmail.org/thread/6yv2fyrs4jlepmmr>>
> >>
> >>
> >>
> >
>
>

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Vinod Kumar Vavilapalli]

But, everyone please do continue your sanity checking on RC0 in case there are 
more issues to be fixed.

Thanks
+Vinod

> On Jul 26, 2016, at 12:11 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> Thanks Daniel and Wei.
> 
> I think these are worth fixing, I’m withdrawing this RC. Will look at fixing 
> these issues and roll a new candidate with the fixes as soon as possible.
> 
> Thanks
> +Vinod
> 
>> On Jul 26, 2016, at 11:05 AM, Wei-Chiu Chuang > > wrote:
>> 
>> I noticed two issues:
>> 
>> (1) I ran hadoop checknative, but it seems the binary tarball was not 
>> compiled with native library for Linux. On the contrary, the Hadoop built 
>> from source tarball with maven -Pnative can find the native libraries on the 
>> same host.
>> 
>> (2) I noticed that the release dates in CHANGES.txt in tag release-2.7.3-RC0 
>> are set to Release 2.7.3 - 2016-07-27.
>> However, the release dates in CHANGES.txt in the source and binary tar balls 
>> are set to Release 2.7.3 - 2016-08-01. This is probably a non-issue though.
>> 
>> * Downloaded source and binary.
>> * Verified signature.
>> * Verified checksum.
>> * Built from source using 64-bit Java 7 (1.7.0.75) and 8 (1.8.0.05). Both 
>> went fine.
>> * Ran hadoop checknative
>> 
>> On Tue, Jul 26, 2016 at 9:12 AM, Rushabh Shah 
>> > 
>> wrote:
>> Thanks Vinod for all the release work !
>> +1 (non-binding).
>> * Downloaded from source and built it.* Deployed a pseudo distributed 
>> cluster.
>> * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything works 
>> fine.
>> 
>> 
>> On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli 
>> > wrote:
>> 
>> 
>>  Hi all,
>> 
>> I've created a release candidate RC0 for Apache Hadoop 2.7.3.
>> 
>> As discussed before, this is the next maintenance release to follow up 2.7.2.
>> 
>> The RC is available for validation at: 
>> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ 
>>  
>> > >
>> 
>> The RC tag in git is: release-2.7.3-RC0
>> 
>> The maven artifacts are available via repository.apache.org 
>>  > > at 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1040/ 
>>  
>> > >
>> 
>> The release-notes are inside the tar-balls at location 
>> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I 
>> hosted this at 
>> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html 
>>  
>> > > for 
>> your quick perusal.
>> 
>> As you may have noted, a very long fix-cycle for the License & Notice issues 
>> (HADOOP-12893) caused 2.7.3 (along with every other Hadoop release) to slip 
>> by quite a bit. This release's related discussion thread is linked below: 
>> [1].
>> 
>> Please try the release and vote; the vote will run for the usual 5 days.
>> 
>> Thanks,
>> Vinod
>> 
>> [1]: 2.7.3 release plan: 
>> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html 
>>  
>> > >
>> 
>>
>> 
> 

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Vinod Kumar Vavilapalli]

Thanks Daniel and Wei.

I think these are worth fixing, I’m withdrawing this RC. Will look at fixing 
these issues and roll a new candidate with the fixes as soon as possible.

Thanks
+Vinod

> On Jul 26, 2016, at 11:05 AM, Wei-Chiu Chuang  wrote:
> 
> I noticed two issues:
> 
> (1) I ran hadoop checknative, but it seems the binary tarball was not 
> compiled with native library for Linux. On the contrary, the Hadoop built 
> from source tarball with maven -Pnative can find the native libraries on the 
> same host.
> 
> (2) I noticed that the release dates in CHANGES.txt in tag release-2.7.3-RC0 
> are set to Release 2.7.3 - 2016-07-27.
> However, the release dates in CHANGES.txt in the source and binary tar balls 
> are set to Release 2.7.3 - 2016-08-01. This is probably a non-issue though.
> 
> * Downloaded source and binary.
> * Verified signature.
> * Verified checksum.
> * Built from source using 64-bit Java 7 (1.7.0.75) and 8 (1.8.0.05). Both 
> went fine.
> * Ran hadoop checknative
> 
> On Tue, Jul 26, 2016 at 9:12 AM, Rushabh Shah  > wrote:
> Thanks Vinod for all the release work !
> +1 (non-binding).
> * Downloaded from source and built it.* Deployed a pseudo distributed cluster.
> * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything works 
> fine.
> 
> 
> On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli 
> > wrote:
> 
> 
>  Hi all,
> 
> I've created a release candidate RC0 for Apache Hadoop 2.7.3.
> 
> As discussed before, this is the next maintenance release to follow up 2.7.2.
> 
> The RC is available for validation at: 
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ 
>  
>  >
> 
> The RC tag in git is: release-2.7.3-RC0
> 
> The maven artifacts are available via repository.apache.org 
>   > at 
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/ 
>  
>  >
> 
> The release-notes are inside the tar-balls at location 
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I hosted 
> this at http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html 
>  
>  > for 
> your quick perusal.
> 
> As you may have noted, a very long fix-cycle for the License & Notice issues 
> (HADOOP-12893) caused 2.7.3 (along with every other Hadoop release) to slip 
> by quite a bit. This release's related discussion thread is linked below: [1].
> 
> Please try the release and vote; the vote will run for the usual 5 days.
> 
> Thanks,
> Vinod
> 
> [1]: 2.7.3 release plan: 
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html 
>  
>  >
> 
>
> 

[DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

[Vinod Kumar Vavilapalli]

Forking the thread to make sure it attracts enough eye-balls. The earlier one 
was about 3.0.0 specifically and I don’t think enough people were watching that.

I’ll try to summarize a bit.

# Today’s state of release numbering and ordering:
So far, all the releases we have done, we have followed a simple timeline 
ordering. By this I mean that we always lined up releases one after another. 
Due to this, it was relatively simple to follow the causal ordering of 
releases, and we could answer ordering questions like “when was this patch 
first committed”.

The first time this started getting hairy was with parallel 2.6.x and 2.7.x 
releases. We managed the confusion by ordering them one after another: 2.7.1 -> 
2.6.2 -> 2.6.3 -> 2.7.2 -> 2.6.4 -> 2.7.3. This worked okay with two concurrent 
releases. 

Yes, by doing this, we could no longer answer questions by simply looking 
at the release numbers. But Sangjin / Junping / myself who did these 2.x 
releases ordered them one after another to avoid further confusion to 
developers and still retain the ability to just go back, look at the history 
and quickly answer the question of "what happened before and what happened 
after”. It wasn’t perfect, but we did what we did to keep it under control.

# What’s coming
Obviously, with 2.8.0 and 3.0.0-alpha1 release trains starting, this 
confusion goes to a whole new level. We’ll have 2.6.x, 2.7.x, 2.8.x, 3.0.0.x 
(and 2.9.x if we chose to make one) and following the ordering becomes a 
nightmare. The related problems that were discussed in the original thread was 
about how we mark fix-versions for patches, and how we generate change-logs - 
the answers for both of these follow the solution to the root problem of 
concurrent releases.

# Proposals?
There were some discussions about semantic versioning in the thread form 
which I forked this one from, but it’s not clear to me how semantic versioning 
supports concurrent release trains.

IAC, it’s time we look at this afresh and if need be, make some new rules 
before we make more of these releases and make it that much harder to follow 
for everyone.

Thanks
+Vinod
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Setting JIRA fix versions for 3.0.0 releases

[Wangda Tan]

Hi all,

I'm trying to generate JDiff for sub projects of Hadoop, some updates:
*- Common*: JDiff cannot be generated , filed
https://issues.apache.org/jira/browse/HADOOP-13428 and debugging that.
- *HDFS*: It pointed to a older version (2.6.0), we need to upgrade it to
the latest stable release (probably 2.7.3, it is mostly done). filed
https://issues.apache.org/jira/browse/HDFS-10692 and will work on that
- *YARN*: JDiff can be generated, we need to do analysis for it.
- *MR*: JDiff cannot be generated,
https://issues.apache.org/jira/browse/MAPREDUCE-6310 is tracking it.

I'm currently actively working on these items, any helps will be
appreciated.

Thanks,
Wangda

On Tue, Jul 26, 2016 at 11:16 AM, Vinod Kumar Vavilapalli <
vino...@apache.org> wrote:

> +1
>
> Thanks
> +Vinod
>
> On Jul 26, 2016, at 7:39 AM, Wangda Tan  wrote:
>
> lets try to use both jdiff and the new tool and compare results because
> this is the first time with the new tool.
>
> Appreciate your time to help us about this effort!
>
>
>

Re: Setting JIRA fix versions for 3.0.0 releases

[Vinod Kumar Vavilapalli]

+1

Thanks
+Vinod

> On Jul 26, 2016, at 7:39 AM, Wangda Tan  wrote:
> 
> lets try to use both jdiff and the new tool and compare results because this 
> is the first time with the new tool.
> 
> Appreciate your time to help us about this effort!

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Wei-Chiu Chuang]

I noticed two issues:

(1) I ran hadoop checknative, but it seems the binary tarball was not
compiled with native library for Linux. On the contrary, the Hadoop built
from source tarball with maven -Pnative can find the native libraries on
the same host.

(2) I noticed that the release dates in CHANGES.txt in tag
release-2.7.3-RC0 are set to Release 2.7.3 - 2016-07-27.
However, the release dates in CHANGES.txt in the source and binary tar
balls are set to Release 2.7.3 - 2016-08-01. This is probably a non-issue
though.

* Downloaded source and binary.
* Verified signature.
* Verified checksum.
* Built from source using 64-bit Java 7 (1.7.0.75) and 8 (1.8.0.05). Both
went fine.
* Ran hadoop checknative

On Tue, Jul 26, 2016 at 9:12 AM, Rushabh Shah <
rusha...@yahoo-inc.com.invalid> wrote:

> Thanks Vinod for all the release work !
> +1 (non-binding).
> * Downloaded from source and built it.* Deployed a pseudo distributed
> cluster.
> * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything
> works fine.
>
>
> On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org> wrote:
>
>
>  Hi all,
>
> I've created a release candidate RC0 for Apache Hadoop 2.7.3.
>
> As discussed before, this is the next maintenance release to follow up
> 2.7.2.
>
> The RC is available for validation at:
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>
>
> The RC tag in git is: release-2.7.3-RC0
>
> The maven artifacts are available via repository.apache.org <
> http://repository.apache.org/> at
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/ <
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/>
>
> The release-notes are inside the tar-balls at location
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I
> hosted this at
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html> for
> your quick perusal.
>
> As you may have noted, a very long fix-cycle for the License & Notice
> issues (HADOOP-12893) caused 2.7.3 (along with every other Hadoop release)
> to slip by quite a bit. This release's related discussion thread is linked
> below: [1].
>
> Please try the release and vote; the vote will run for the usual 5 days.
>
> Thanks,
> Vinod
>
> [1]: 2.7.3 release plan:
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html <
> http://markmail.org/thread/6yv2fyrs4jlepmmr>
>
>
>

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Daniel Templeton]

I just downloaded the build tarball and deployed it on a 2-node 
cluster.  It looks to me like it's compiled for the wrong platform:


# file /usr/lib/hadoop/bin/container-executor
/usr/lib/hadoop/bin/container-executor: setuid setgid Mach-O 64-bit 
executable


I'm also seeing the no-native-libraries warnings.

Daniel

On 7/26/16 6:12 PM, Rushabh Shah wrote:

Thanks Vinod for all the release work !
+1 (non-binding).
* Downloaded from source and built it.* Deployed a pseudo distributed cluster.
* Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything works 
fine.
  


 On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli 
 wrote:
  


  Hi all,

I've created a release candidate RC0 for Apache Hadoop 2.7.3.

As discussed before, this is the next maintenance release to follow up 2.7.2.

The RC is available for validation at: 
http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ 


The RC tag in git is: release-2.7.3-RC0

The maven artifacts are available via repository.apache.org 
 at 
https://repository.apache.org/content/repositories/orgapachehadoop-1040/ 


The release-notes are inside the tar-balls at location 
hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I hosted this at 
http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html 
 for your 
quick perusal.

As you may have noted, a very long fix-cycle for the License & Notice issues 
(HADOOP-12893) caused 2.7.3 (along with every other Hadoop release) to slip by 
quite a bit. This release's related discussion thread is linked below: [1].

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[1]: 2.7.3 release plan: 
https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html 







-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Rushabh Shah]

Thanks Vinod for all the release work !
+1 (non-binding).
* Downloaded from source and built it.* Deployed a pseudo distributed cluster.
* Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything works 
fine.
 

On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli 
 wrote:
 

 Hi all,

I've created a release candidate RC0 for Apache Hadoop 2.7.3.

As discussed before, this is the next maintenance release to follow up 2.7.2.

The RC is available for validation at: 
http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ 


The RC tag in git is: release-2.7.3-RC0

The maven artifacts are available via repository.apache.org 
 at 
https://repository.apache.org/content/repositories/orgapachehadoop-1040/ 


The release-notes are inside the tar-balls at location 
hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I hosted 
this at http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html 
 for your 
quick perusal.

As you may have noted, a very long fix-cycle for the License & Notice issues 
(HADOOP-12893) caused 2.7.3 (along with every other Hadoop release) to slip by 
quite a bit. This release's related discussion thread is linked below: [1].

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[1]: 2.7.3 release plan: 
https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html 


   

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Masatake Iwasaki]

Thanks for putting this up, Vinod.

+1 (non-binding)

* verified signature and mds of source and binary tarball
* built from source tarball on CentOS 6
* built site documentation
* deployed 3-node cluster with NN-HA and RM-HA, ran example jobs
* built rpms by using Bigtop, deployed 3-node cluster on docker and ran 
smoke-tests of hdfs, yarn and mapreduce


Masatake Iwasaki

On 7/23/16 11:15, Vinod Kumar Vavilapalli wrote:

Hi all,

I've created a release candidate RC0 for Apache Hadoop 2.7.3.

As discussed before, this is the next maintenance release to follow up 2.7.2.

The RC is available for validation at: 
http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ 


The RC tag in git is: release-2.7.3-RC0

The maven artifacts are available via repository.apache.org 
 at 
https://repository.apache.org/content/repositories/orgapachehadoop-1040/ 


The release-notes are inside the tar-balls at location 
hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I hosted this at 
http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html 
 for your 
quick perusal.

As you may have noted, a very long fix-cycle for the License & Notice issues 
(HADOOP-12893) caused 2.7.3 (along with every other Hadoop release) to slip by 
quite a bit. This release's related discussion thread is linked below: [1].

Please try the release and vote; the vote will run for the usual 5 days.

Thanks,
Vinod

[1]: 2.7.3 release plan: 
https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html 




-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/

[Jul 25, 2016 1:45:03 PM] (stevel) HADOOP-13406 S3AFileSystem: Consider reusing 
filestatus in delete() and
[Jul 25, 2016 2:50:23 PM] (stevel) HADOOP-13188 S3A file-create should throw 
error rather than overwrite
[Jul 25, 2016 9:54:48 PM] (jlowe) MAPREDUCE-6744. Increase timeout on TestDFSIO 
tests. Contributed by Eric
[Jul 25, 2016 11:37:50 PM] (cdouglas) YARN-5164. Use plan RLE to improve 
CapacityOverTimePolicy efficiency
[Jul 26, 2016 1:41:13 AM] (jing9) HDFS-10688. BPServiceActor may run into a 
tight loop for sending block
[Jul 26, 2016 1:48:21 AM] (iwasakims) HDFS-10671. Fix typo in 
HdfsRollingUpgrade.md. Contributed by Yiqun Lin.
[Jul 26, 2016 1:50:59 AM] (shv) HDFS-10301. Interleaving processing of storages 
from repeated block
[Jul 26, 2016 5:24:24 AM] (brahma) HDFS-10668. Fix intermittently failing UT




-1 overall


The following subsystems voted -1:
asflicense unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.balancer.TestBalancer 
   hadoop.hdfs.TestDFSShell 
   hadoop.hdfs.TestDecommissionWithStriped 
   hadoop.hdfs.server.datanode.TestFsDatasetCache 
   hadoop.yarn.server.nodemanager.TestDirectoryCollection 
   hadoop.yarn.server.TestMiniYarnClusterNodeUtilization 
   hadoop.yarn.server.TestContainerManagerSecurity 
   hadoop.yarn.client.api.impl.TestYarnClient 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/diff-compile-javac-root.txt
  [172K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/diff-checkstyle-root.txt
  [16M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/diff-patch-pylint.txt
  [16K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/diff-patch-shelldocs.txt
  [16K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/whitespace-eol.txt
  [12M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/whitespace-tabs.txt
  [1.3M]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/diff-javadoc-javadoc-root.txt
  [2.3M]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [436K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [36K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
  [268K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt
  [124K]

   asflicense:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/artifact/out/patch-asflicense-problems.txt
  [4.0K]

Powered by Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org



-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Rohith Sharma K S]

+1(non-binding)

Downloaded and built from source
Cluster installed in 3 nodes and verified running simple MR jobs. 
Verified for RM HA , RM work preserving restart with CapacityScheduler


Thanks & Regards
Rohith Sharma K S

> On Jul 26, 2016, at 6:50 PM, Vinayakumar B  wrote:
> 
> +1 (binding)
> 
> 1. Downloaded and Built from branch-2.7.3
> 2. Started up HDFS and YARN in Single Node cluster.
> 3. Ran WordCount job multiple times and Success.
> 4. Verified the "Release Notes" available at the URL mentioned by Vinod.
> 
> 
> Apart from that, 
> Faced same issues as Andrew wang, while running the WordCount job first time 
> in my new Ubuntu installation, without 'configuring the shuffle handler 
> properly'. Whole session got logged by closing all other applications open. 
> After configuring the shuffle handler properly, job was successful though.
> 
> -Vinay
> 
> -Original Message-
> From: Andrew Wang [mailto:andrew.w...@cloudera.com] 
> Sent: 26 July 2016 00:22
> To: Karthik Kambatla 
> Cc: larry mccay ; Vinod Kumar Vavilapalli 
> ; common-...@hadoop.apache.org; 
> hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org; 
> mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.7.3 RC0
> 
> I'll also add that, as a YARN newbie, I did hit two usability issues. These 
> are very unlikely to be regressions, and I can file JIRAs if they seem 
> fixable.
> 
> * I didn't have SSH to localhost set up (new laptop), and when I tried to run 
> the Pi job, it'd exit my window manager session. I feel there must be a more 
> developer-friendly solution here.
> * If you start the NodeManager and not the RM, the NM has a handler for 
> SIGTERM and SIGINT that blocked my Ctrl-C and kill attempts during startup.
> I had to kill -9 it.
> 
> On Mon, Jul 25, 2016 at 11:44 AM, Andrew Wang 
> wrote:
> 
>> I got asked this off-list, so as a reminder, only PMC votes are 
>> binding on releases. Everyone is encouraged to vote on releases though!
>> 
>> +1 (binding)
>> 
>> * Downloaded source, built
>> * Started up HDFS and YARN
>> * Ran Pi job which as usual returned 4, and a little teragen
>> 
>> On Mon, Jul 25, 2016 at 11:08 AM, Karthik Kambatla 
>> 
>> wrote:
>> 
>>> +1 (binding)
>>> 
>>> * Downloaded and build from source
>>> * Checked LICENSE and NOTICE
>>> * Pseudo-distributed cluster with FairScheduler
>>> * Ran MR and HDFS tests
>>> * Verified basic UI
>>> 
>>> On Sun, Jul 24, 2016 at 1:07 PM, larry mccay  wrote:
>>> 
 +1 binding
 
 * downloaded and built from source
 * checked LICENSE and NOTICE files
 * verified signatures
 * ran standalone tests
 * installed pseudo-distributed instance on my mac
 * ran through HDFS and mapreduce tests
 * tested credential command
 * tested webhdfs access through Apache Knox
 
 
 On Fri, Jul 22, 2016 at 10:15 PM, Vinod Kumar Vavilapalli < 
 vino...@apache.org> wrote:
 
> Hi all,
> 
> I've created a release candidate RC0 for Apache Hadoop 2.7.3.
> 
> As discussed before, this is the next maintenance release to 
> follow up 2.7.2.
> 
> The RC is available for validation at:
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ < 
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>
> 
> The RC tag in git is: release-2.7.3-RC0
> 
> The maven artifacts are available via repository.apache.org < 
> http://repository.apache.org/> at
> 
>>> https://repository.apache.org/content/repositories/orgapachehadoop-10
>>> 40/
 <
> 
>>> https://repository.apache.org/content/repositories/orgapachehadoop-10
>>> 40/
> 
> 
> The release-notes are inside the tar-balls at location 
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.ht
> ml. I hosted this at 
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.htm
> l < 
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.h
> tml>
 for
> your quick perusal.
> 
> As you may have noted, a very long fix-cycle for the License & 
> Notice issues (HADOOP-12893) caused 2.7.3 (along with every other 
> Hadoop
 release)
> to slip by quite a bit. This release's related discussion thread 
> is
 linked
> below: [1].
> 
> Please try the release and vote; the vote will run for the usual 
> 5
>>> days.
> 
> Thanks,
> Vinod
> 
> [1]: 2.7.3 release plan:
> 
>>> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.ht
>>> ml
 <
> http://markmail.org/thread/6yv2fyrs4jlepmmr>
 
>>> 
>> 
>> 
> 
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: 

RE: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Vinayakumar B]

+1 (binding)

1. Downloaded and Built from branch-2.7.3
2. Started up HDFS and YARN in Single Node cluster.
3. Ran WordCount job multiple times and Success.
4. Verified the "Release Notes" available at the URL mentioned by Vinod.


Apart from that, 
Faced same issues as Andrew wang, while running the WordCount job first time in 
my new Ubuntu installation, without 'configuring the shuffle handler properly'. 
Whole session got logged by closing all other applications open. After 
configuring the shuffle handler properly, job was successful though.

-Vinay

-Original Message-
From: Andrew Wang [mailto:andrew.w...@cloudera.com] 
Sent: 26 July 2016 00:22
To: Karthik Kambatla 
Cc: larry mccay ; Vinod Kumar Vavilapalli 
; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
yarn-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

I'll also add that, as a YARN newbie, I did hit two usability issues. These are 
very unlikely to be regressions, and I can file JIRAs if they seem fixable.

* I didn't have SSH to localhost set up (new laptop), and when I tried to run 
the Pi job, it'd exit my window manager session. I feel there must be a more 
developer-friendly solution here.
* If you start the NodeManager and not the RM, the NM has a handler for SIGTERM 
and SIGINT that blocked my Ctrl-C and kill attempts during startup.
I had to kill -9 it.

On Mon, Jul 25, 2016 at 11:44 AM, Andrew Wang 
wrote:

> I got asked this off-list, so as a reminder, only PMC votes are 
> binding on releases. Everyone is encouraged to vote on releases though!
>
> +1 (binding)
>
> * Downloaded source, built
> * Started up HDFS and YARN
> * Ran Pi job which as usual returned 4, and a little teragen
>
> On Mon, Jul 25, 2016 at 11:08 AM, Karthik Kambatla 
> 
> wrote:
>
>> +1 (binding)
>>
>> * Downloaded and build from source
>> * Checked LICENSE and NOTICE
>> * Pseudo-distributed cluster with FairScheduler
>> * Ran MR and HDFS tests
>> * Verified basic UI
>>
>> On Sun, Jul 24, 2016 at 1:07 PM, larry mccay  wrote:
>>
>> > +1 binding
>> >
>> > * downloaded and built from source
>> > * checked LICENSE and NOTICE files
>> > * verified signatures
>> > * ran standalone tests
>> > * installed pseudo-distributed instance on my mac
>> > * ran through HDFS and mapreduce tests
>> > * tested credential command
>> > * tested webhdfs access through Apache Knox
>> >
>> >
>> > On Fri, Jul 22, 2016 at 10:15 PM, Vinod Kumar Vavilapalli < 
>> > vino...@apache.org> wrote:
>> >
>> > > Hi all,
>> > >
>> > > I've created a release candidate RC0 for Apache Hadoop 2.7.3.
>> > >
>> > > As discussed before, this is the next maintenance release to 
>> > > follow up 2.7.2.
>> > >
>> > > The RC is available for validation at:
>> > > http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ < 
>> > > http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>
>> > >
>> > > The RC tag in git is: release-2.7.3-RC0
>> > >
>> > > The maven artifacts are available via repository.apache.org < 
>> > > http://repository.apache.org/> at
>> > >
>> https://repository.apache.org/content/repositories/orgapachehadoop-10
>> 40/
>> > <
>> > >
>> https://repository.apache.org/content/repositories/orgapachehadoop-10
>> 40/
>> > >
>> > >
>> > > The release-notes are inside the tar-balls at location 
>> > > hadoop-common-project/hadoop-common/src/main/docs/releasenotes.ht
>> > > ml. I hosted this at 
>> > > http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.htm
>> > > l < 
>> > > http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.h
>> > > tml>
>> > for
>> > > your quick perusal.
>> > >
>> > > As you may have noted, a very long fix-cycle for the License & 
>> > > Notice issues (HADOOP-12893) caused 2.7.3 (along with every other 
>> > > Hadoop
>> > release)
>> > > to slip by quite a bit. This release's related discussion thread 
>> > > is
>> > linked
>> > > below: [1].
>> > >
>> > > Please try the release and vote; the vote will run for the usual 
>> > > 5
>> days.
>> > >
>> > > Thanks,
>> > > Vinod
>> > >
>> > > [1]: 2.7.3 release plan:
>> > >
>> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.ht
>> ml
>> > <
>> > > http://markmail.org/thread/6yv2fyrs4jlepmmr>
>> >
>>
>
>

Re: Setting JIRA fix versions for 3.0.0 releases

[Sean Busbey]

Just so I don't waste time chasing my tail, should I interpret this
email and the associated JIRA as the PMC preferring I not spend
volunteer time providing a compatibility breakdown as previously
discussed?

On Mon, Jul 25, 2016 at 7:54 PM, Wangda Tan  wrote:
> I just filed ticket https://issues.apache.org/jira/browse/HADOOP-13423 to
> track running JDIFF on trunk and analyze results for Hadoop-common. I will
> work on that and keep the JIRA and this thread updated. We need to do the
> same work for YARN/MR/HDFS.
>
> On Mon, Jul 25, 2016 at 5:47 PM, Wangda Tan  wrote:
>>
>> I agree with what Vinod mentioned: we need to revisit all incompatible
>> changes and revert unnecessary ones. Even if we don't have any compatibility
>> guarantees between 2.x and 3.x. But make user to be less frustrated while
>> trying 3.x is always a better option to me.
>>
>> To achieve this we need to run jdiff for trunk and look at results. I
>> would suggest to do this before 3.0.0-alpha1 release.
>>
>> In addition, for people who will try this 3-alpha1 release artifacts, a
>> guide about migration from 2.x to 3.x will be very helpful, and it can also
>> help for people to better understand what have changed (Just like
>> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html)
>>
>> Thoughts?
>>
>> Thanks,
>> Wangda
>>
>>
>> On Thu, Jul 21, 2016 at 2:41 PM, Sean Busbey  wrote:
>>>
>>> On Thu, Jul 21, 2016 at 4:32 PM, Vinod Kumar Vavilapalli
>>>  wrote:
>>> >> I really, really want a 3.0.0-alpha1 ASAP, since it's basically
>>> >> impossible for downstreams to test incompat changes and new features 
>>> >> without
>>> >> a release artifact. I've been doing test builds, and branch-3.0.0-alpha1 
>>> >> is
>>> >> ready for an RC besides possibly this fix version issue.
>>> >
>>> > Not arguing against the need for an alpha release, the question is if
>>> > it can wait till after 2.8 gets done.
>>> >
>>> > Orthogonally, do we have a report of the incompatible changes? Like the
>>> > one I generated for some of the branch-2 releases using late jdiff work 
>>> > from
>>> > Li Lu etc. We should do this and fix any inadvertant incompatibilities.
>>> > Without seeing this list of incompatibilities, why even make an alpha
>>> > release and force downstream components to discover issues what can be
>>> > identified through running reports.
>>> >
>>>
>>> I can come up with this, atleast for Source / Binary API compatibility,
>>> provided folks don't mind if I use the Java API Compliance Checker[1]
>>> instead of jdiff.
>>>
>>> I'm already familiar with quickly using it, esp with Audience
>>> Annotations from my work in HBase.
>>>
>>> Do you want this check from some particular branch-2 release? It
>>> matters since the releases along branch-2 have themselves had some
>>> noise[2].
>>>
>>> [1]: https://github.com/lvc/japi-compliance-checker
>>> [2]: http://abi-laboratory.pro/java/tracker/timeline/hadoop/
>>>
>>> --
>>> busbey
>>>
>>> -
>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>>
>>
>



-- 
busbey

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Setting JIRA fix versions for 3.0.0 releases

[Sean Busbey]

Yes, the Java API Compliance Checker allows specifying Annotations to
pare down where incompatible changes happen. It was added some time
ago based on feedback from the Apache HBase project.

The limitations I've found are: 1) at least earlier versions only
supported annotations at the class level (which prevents things like
VisibleForTesting), 2) sometimes it will include more restricted
classes if they are used in less restrictive APIs (e.g. if a IA.Public
class makes use of a IA.Private class in a method signature).

At least when we've used it in HBase, these limitations have been very
easy to spot an explain in a small amount of text. I expect I will be
able to do the same with Hadoop. If we'd like to automate this, the
author has been very responsive to feature requests thus far.


On Mon, Jul 25, 2016 at 3:47 PM, Vinod Kumar Vavilapalli
 wrote:
> Actually, I wouldn’t trust this report as it stands today at all.
>
> I quickly glanced the report, looking for what it highlights as
> incompatible. But the ones that I saw have private / visible for testing
> annotations. Other than acting as useless evidence for those lashing out on
> branch-2, this won’t do much good.
>
> Whenever we start working towards switching to this tool, it should
> incorporate the same exclude-annotations logic that the jdiff code-path does
> today. Do you think that is possible?
>
> Thanks
> +Vinod
>
> On Jul 22, 2016, at 4:53 PM, Vinod Kumar Vavilapalli 
> wrote:
>
> Do you want this check from some particular branch-2 release? It
> matters since the releases along branch-2 have themselves had some
> noise[2].
>
> [1]: https://github.com/lvc/japi-compliance-checker
> 
> [2]: http://abi-laboratory.pro/java/tracker/timeline/hadoop/
> 
>
> --
> busbey
>
>



-- 
busbey

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Rakesh Radhakrishnan]

Thank you Vinod.

+1 (non-binding)

- downloaded and built from source
- deployed HDFS-HA cluster and tested few switching behaviors
- executed few hdfs commands from command line
- viewed basic UI
- ran HDFS/Common unit tests
- checked LICENSE and NOTICE files

Regards,
Rakesh
Intel

On Tue, Jul 26, 2016 at 11:36 AM, Zhihai Xu  wrote:

> Thanks Vinod.
>
> +1 (non-binding)
>
> * Downloaded and built from source
> * Checked LICENSE and NOTICE
> * Deployed a pseudo cluster
> * Ran through MR and HDFS tests
> * verified basic HDFS operations and Pi job.
>
> Zhihai
>
> On Fri, Jul 22, 2016 at 7:15 PM, Vinod Kumar Vavilapalli <
> vino...@apache.org
> > wrote:
>
> > Hi all,
> >
> > I've created a release candidate RC0 for Apache Hadoop 2.7.3.
> >
> > As discussed before, this is the next maintenance release to follow up
> > 2.7.2.
> >
> > The RC is available for validation at:
> > http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> > http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>
> >
> > The RC tag in git is: release-2.7.3-RC0
> >
> > The maven artifacts are available via repository.apache.org <
> > http://repository.apache.org/> at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1040/
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1040/
> >
> >
> > The release-notes are inside the tar-balls at location
> > hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I
> > hosted this at
> > http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html <
> > http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html>
> for
> > your quick perusal.
> >
> > As you may have noted, a very long fix-cycle for the License & Notice
> > issues (HADOOP-12893) caused 2.7.3 (along with every other Hadoop
> release)
> > to slip by quite a bit. This release's related discussion thread is
> linked
> > below: [1].
> >
> > Please try the release and vote; the vote will run for the usual 5 days.
> >
> > Thanks,
> > Vinod
> >
> > [1]: 2.7.3 release plan:
> > https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html
> <
> > http://markmail.org/thread/6yv2fyrs4jlepmmr>
>

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

[Zhihai Xu]

Thanks Vinod.

+1 (non-binding)

* Downloaded and built from source
* Checked LICENSE and NOTICE
* Deployed a pseudo cluster
* Ran through MR and HDFS tests
* verified basic HDFS operations and Pi job.

Zhihai

On Fri, Jul 22, 2016 at 7:15 PM, Vinod Kumar Vavilapalli  wrote:

> Hi all,
>
> I've created a release candidate RC0 for Apache Hadoop 2.7.3.
>
> As discussed before, this is the next maintenance release to follow up
> 2.7.2.
>
> The RC is available for validation at:
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/ <
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/>
>
> The RC tag in git is: release-2.7.3-RC0
>
> The maven artifacts are available via repository.apache.org <
> http://repository.apache.org/> at
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/ <
> https://repository.apache.org/content/repositories/orgapachehadoop-1040/>
>
> The release-notes are inside the tar-balls at location
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I
> hosted this at
> http://home.apache.org/~vinodkv/hadoop-2.7.3-RC0/releasenotes.html <
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC1/releasenotes.html> for
> your quick perusal.
>
> As you may have noted, a very long fix-cycle for the License & Notice
> issues (HADOOP-12893) caused 2.7.3 (along with every other Hadoop release)
> to slip by quite a bit. This release's related discussion thread is linked
> below: [1].
>
> Please try the release and vote; the vote will run for the usual 5 days.
>
> Thanks,
> Vinod
>
> [1]: 2.7.3 release plan:
> https://www.mail-archive.com/hdfs-dev%40hadoop.apache.org/msg24439.html <
> http://markmail.org/thread/6yv2fyrs4jlepmmr>