Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-16 Thread Naganarasimha Garla
+1 (binding) Apologies for one more late vote!

- Built from source and also installed the release tar.
- Verified all Node Labels and node .
-  Ran sample jobs and verified UI

Thanks and Regards,
+ Naga

On Wed, Jan 16, 2019 at 10:34 PM Rohith Sharma K S <
rohithsharm...@apache.org> wrote:

> +1 (binding) Apologies for late vote!
>
> - Built from source for hbase profile=2.0 and deployed RM HA cluster along
> with ATSv2.0 enabled.
> - Ran sample jobs and verified all ATSv2.0 data are publishing.
> - Verified UI2 for ATSv2.0 data!
>
> -Rohith Sharma K S
>
> On Wed, 16 Jan 2019 at 07:00, Sunil G  wrote:
>
> > Thanks everyone for helping to vote this release!
> >
> > With 7 binding votes, 10 non-binding votes and no veto, this vote stands
> >  passed,
> > I'm going to work on staging the release.
> >
> > Thanks,
> > Sunil
> >
> > On Tue, Jan 15, 2019 at 9:59 PM Weiwei Yang  wrote:
> >
> > > +1 (binding)
> > >
> > > - Setup a cluster, run teragen/terasort jobs
> > > - Verified general readability of documentation (titles/navigations)
> > > - Run some simple yarn commands: app/applicationattempt/container
> > > - Checked restful APIs: RM cluster/metrics/scheduler/nodes, NM
> > > node/apps/container
> > > - Verified simple failover scenario
> > > - Submitted distributed shell apps with affinity/anti-affinity
> > constraints
> > > - Configured conf based node attribute provider, alter attribute values
> > > and verified the change
> > > - Verified CLI add/list/remove node-attributes, submitted app with
> simple
> > > node-attribute constraint
> > >
> > > --
> > > Weiwei
> > >
> >
>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-16 Thread Rohith Sharma K S
+1 (binding) Apologies for late vote!

- Built from source for hbase profile=2.0 and deployed RM HA cluster along
with ATSv2.0 enabled.
- Ran sample jobs and verified all ATSv2.0 data are publishing.
- Verified UI2 for ATSv2.0 data!

-Rohith Sharma K S

On Wed, 16 Jan 2019 at 07:00, Sunil G  wrote:

> Thanks everyone for helping to vote this release!
>
> With 7 binding votes, 10 non-binding votes and no veto, this vote stands
>  passed,
> I'm going to work on staging the release.
>
> Thanks,
> Sunil
>
> On Tue, Jan 15, 2019 at 9:59 PM Weiwei Yang  wrote:
>
> > +1 (binding)
> >
> > - Setup a cluster, run teragen/terasort jobs
> > - Verified general readability of documentation (titles/navigations)
> > - Run some simple yarn commands: app/applicationattempt/container
> > - Checked restful APIs: RM cluster/metrics/scheduler/nodes, NM
> > node/apps/container
> > - Verified simple failover scenario
> > - Submitted distributed shell apps with affinity/anti-affinity
> constraints
> > - Configured conf based node attribute provider, alter attribute values
> > and verified the change
> > - Verified CLI add/list/remove node-attributes, submitted app with simple
> > node-attribute constraint
> >
> > --
> > Weiwei
> >
>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-16 Thread Sunil G
Thanks everyone for helping to vote this release!

With 7 binding votes, 10 non-binding votes and no veto, this vote stands
 passed,
I'm going to work on staging the release.

Thanks,
Sunil

On Tue, Jan 15, 2019 at 9:59 PM Weiwei Yang  wrote:

> +1 (binding)
>
> - Setup a cluster, run teragen/terasort jobs
> - Verified general readability of documentation (titles/navigations)
> - Run some simple yarn commands: app/applicationattempt/container
> - Checked restful APIs: RM cluster/metrics/scheduler/nodes, NM
> node/apps/container
> - Verified simple failover scenario
> - Submitted distributed shell apps with affinity/anti-affinity constraints
> - Configured conf based node attribute provider, alter attribute values
> and verified the change
> - Verified CLI add/list/remove node-attributes, submitted app with simple
> node-attribute constraint
>
> --
> Weiwei
>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Brahma Reddy Battula
Sunil thanks for driving this.

+1 ,(binding)

--Built from Source.
--Installed pseudo cluster.
--Ran basic shell commands
--Ran sample jobs pi,wordcount.



On Tue, Jan 8, 2019 at 5:12 PM Sunil G  wrote:

> Hi folks,
>
>
> Thanks to all of you who helped in this release [1] and for helping to vote
> for RC0. I have created second release candidate (RC1) for Apache Hadoop
> 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
>
>
> RC tag in git is release-3.2.0-RC1.
>
>
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
>
>
> This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>


-- 
--Brahma Reddy Battula


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Eric Payne
+1 (binding)

- RM refresh updates values as expected
- Streaming jobs complete successfully
- Moving apps between queues succeeds
- Inter-queue preemption works as expected
- Successfully ran selected yarn unit tests.

===
Eric Payne
===


On Tuesday, January 8, 2019, 5:42:46 AM CST, Sunil G  wrote: 





Hi folks,


Thanks to all of you who helped in this release [1] and for helping to vote
for RC0. I have created second release candidate (RC1) for Apache Hadoop
3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/


RC tag in git is release-3.2.0-RC1.



The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1178/


This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.



3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org



Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Wangda Tan
Manually checked the new repo, looks good, thanks Sunil!  +1

Best,
Wangda

On Tue, Jan 15, 2019 at 2:04 PM Sunil G  wrote:

> Thanks folks for voting.
>
> For the point mentioned from Zoltan, I re-ran the same source and deployed
> to Nexus to avail those missing source.
> https://repository.apache.org/content/repositories/orgapachehadoop-1186/
>
> Please help to cross check the same.
>
> Thanks & Regards
> Sunil
>
> On Tue, Jan 15, 2019 at 10:05 AM Wangda Tan  wrote:
>
>> +1 (Binding).
>>
>> Deployed a local cluster from binary, and ran some sample sanity jobs.
>>
>> Thanks Sunil for driving the release.
>>
>> Best,
>> Wangda
>>
>>
>> On Mon, Jan 14, 2019 at 11:26 AM Virajith Jalaparti 
>> wrote:
>>
>>> Thanks Sunil and others who have worked on the making this release
>>> happen!
>>>
>>> +1 (non-binding)
>>>
>>> - Built from source
>>> - Deployed a pseudo-distributed one node cluster
>>> - Ran basic wordcount, sort, pi jobs
>>> - Basic HDFS/WebHDFS commands
>>> - Ran all the ABFS driver tests against an ADLS Gen 2 account in EAST US
>>>
>>> Non-blockers (AFAICT): The following tests in ABFS (HADOOP-15407) fail:
>>> - For ACLs ({{ITestAzureBlobFilesystemAcl}}) -- However, I believe these
>>> have been fixed in trunk.
>>> - {{
>>> ITestAzureBlobFileSystemE2EScale#testWriteHeavyBytesToFileAcrossThreads}}
>>> fails with an OutOfMemoryError exception. I see the same failure on
>>> trunk as well.
>>>
>>>
>>> On Mon, Jan 14, 2019 at 6:21 AM Elek, Marton  wrote:
>>>
 Thanks Sunil to manage this release.

 +1 (non-binding)

 1. built from the source (with clean local maven repo)
 2. verified signatures + checksum
 3. deployed 3 node cluster to Google Kubernetes Engine with generated
 k8s resources [1]
 4. Executed basic HDFS commands
 5. Executed basic yarn example jobs

 Marton

 [1]: FTR: resources:
 https://github.com/flokkr/k8s/tree/master/examples/hadoop , generator:
 https://github.com/elek/flekszible


 On 1/8/19 12:42 PM, Sunil G wrote:
 > Hi folks,
 >
 >
 > Thanks to all of you who helped in this release [1] and for helping
 to vote
 > for RC0. I have created second release candidate (RC1) for Apache
 Hadoop
 > 3.2.0.
 >
 >
 > Artifacts for this RC are available here:
 >
 > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
 >
 >
 > RC tag in git is release-3.2.0-RC1.
 >
 >
 >
 > The maven artifacts are available via repository.apache.org at
 >
 https://repository.apache.org/content/repositories/orgapachehadoop-1178/
 >
 >
 > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59
 pm PST.
 >
 >
 >
 > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
 > additions
 >
 > are the highlights of this release.
 >
 > 1. Node Attributes Support in YARN
 >
 > 2. Hadoop Submarine project for running Deep Learning workloads on
 YARN
 >
 > 3. Support service upgrade via YARN Service API and CLI
 >
 > 4. HDFS Storage Policy Satisfier
 >
 > 5. Support Windows Azure Storage - Blob file system in Hadoop
 >
 > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
 >
 > 7. Improvements in Router-based HDFS federation
 >
 >
 >
 > Thanks to Wangda, Vinod, Marton for helping me in preparing the
 release.
 >
 > I have done few testing with my pseudo cluster. My +1 to start.
 >
 >
 >
 > Regards,
 >
 > Sunil
 >
 >
 >
 > [1]
 >
 >
 https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
 >
 > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
 (3.2.0)
 > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
 Resolved
 > ORDER BY fixVersion ASC
 >

 -
 To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
 For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org




Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Sunil G
Thanks folks for voting.

For the point mentioned from Zoltan, I re-ran the same source and deployed
to Nexus to avail those missing source.
https://repository.apache.org/content/repositories/orgapachehadoop-1186/

Please help to cross check the same.

Thanks & Regards
Sunil

On Tue, Jan 15, 2019 at 10:05 AM Wangda Tan  wrote:

> +1 (Binding).
>
> Deployed a local cluster from binary, and ran some sample sanity jobs.
>
> Thanks Sunil for driving the release.
>
> Best,
> Wangda
>
>
> On Mon, Jan 14, 2019 at 11:26 AM Virajith Jalaparti 
> wrote:
>
>> Thanks Sunil and others who have worked on the making this release happen!
>>
>> +1 (non-binding)
>>
>> - Built from source
>> - Deployed a pseudo-distributed one node cluster
>> - Ran basic wordcount, sort, pi jobs
>> - Basic HDFS/WebHDFS commands
>> - Ran all the ABFS driver tests against an ADLS Gen 2 account in EAST US
>>
>> Non-blockers (AFAICT): The following tests in ABFS (HADOOP-15407) fail:
>> - For ACLs ({{ITestAzureBlobFilesystemAcl}}) -- However, I believe these
>> have been fixed in trunk.
>> - {{
>> ITestAzureBlobFileSystemE2EScale#testWriteHeavyBytesToFileAcrossThreads}}
>> fails with an OutOfMemoryError exception. I see the same failure on
>> trunk as well.
>>
>>
>> On Mon, Jan 14, 2019 at 6:21 AM Elek, Marton  wrote:
>>
>>> Thanks Sunil to manage this release.
>>>
>>> +1 (non-binding)
>>>
>>> 1. built from the source (with clean local maven repo)
>>> 2. verified signatures + checksum
>>> 3. deployed 3 node cluster to Google Kubernetes Engine with generated
>>> k8s resources [1]
>>> 4. Executed basic HDFS commands
>>> 5. Executed basic yarn example jobs
>>>
>>> Marton
>>>
>>> [1]: FTR: resources:
>>> https://github.com/flokkr/k8s/tree/master/examples/hadoop , generator:
>>> https://github.com/elek/flekszible
>>>
>>>
>>> On 1/8/19 12:42 PM, Sunil G wrote:
>>> > Hi folks,
>>> >
>>> >
>>> > Thanks to all of you who helped in this release [1] and for helping to
>>> vote
>>> > for RC0. I have created second release candidate (RC1) for Apache
>>> Hadoop
>>> > 3.2.0.
>>> >
>>> >
>>> > Artifacts for this RC are available here:
>>> >
>>> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
>>> >
>>> >
>>> > RC tag in git is release-3.2.0-RC1.
>>> >
>>> >
>>> >
>>> > The maven artifacts are available via repository.apache.org at
>>> >
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
>>> >
>>> >
>>> > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm
>>> PST.
>>> >
>>> >
>>> >
>>> > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
>>> > additions
>>> >
>>> > are the highlights of this release.
>>> >
>>> > 1. Node Attributes Support in YARN
>>> >
>>> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>>> >
>>> > 3. Support service upgrade via YARN Service API and CLI
>>> >
>>> > 4. HDFS Storage Policy Satisfier
>>> >
>>> > 5. Support Windows Azure Storage - Blob file system in Hadoop
>>> >
>>> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>>> >
>>> > 7. Improvements in Router-based HDFS federation
>>> >
>>> >
>>> >
>>> > Thanks to Wangda, Vinod, Marton for helping me in preparing the
>>> release.
>>> >
>>> > I have done few testing with my pseudo cluster. My +1 to start.
>>> >
>>> >
>>> >
>>> > Regards,
>>> >
>>> > Sunil
>>> >
>>> >
>>> >
>>> > [1]
>>> >
>>> >
>>> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>>> >
>>> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
>>> (3.2.0)
>>> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
>>> > ORDER BY fixVersion ASC
>>> >
>>>
>>> -
>>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>>>
>>>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Iñigo Goiri
+1 (binding)

- Deployed a cluster with 3 NNs, 3 RMs and 1 DN/NM on Azure
- Tested the Active probe for the Load Balancer in front of the NNs and the
RMs
- Checked the NN, RBF, and RM Web UIs
- Executed a TeraGen, TeraSort  and TeraValidate
- Executed a YARN service with a TensorFlow app on Docker
- Scaled up the cluster to 5 DNs/NMs and executed the tests again
- Checked the HDFS output folders through the Web UI

On Tue, Jan 15, 2019 at 9:05 AM Wangda Tan  wrote:

> +1 (Binding).
>
> Deployed a local cluster from binary, and ran some sample sanity jobs.
>
> Thanks Sunil for driving the release.
>
> Best,
> Wangda
>
>
> On Mon, Jan 14, 2019 at 11:26 AM Virajith Jalaparti 
> wrote:
>
> > Thanks Sunil and others who have worked on the making this release
> happen!
> >
> > +1 (non-binding)
> >
> > - Built from source
> > - Deployed a pseudo-distributed one node cluster
> > - Ran basic wordcount, sort, pi jobs
> > - Basic HDFS/WebHDFS commands
> > - Ran all the ABFS driver tests against an ADLS Gen 2 account in EAST US
> >
> > Non-blockers (AFAICT): The following tests in ABFS (HADOOP-15407) fail:
> > - For ACLs ({{ITestAzureBlobFilesystemAcl}}) -- However, I believe these
> > have been fixed in trunk.
> > - {{
> > ITestAzureBlobFileSystemE2EScale#testWriteHeavyBytesToFileAcrossThreads}}
> > fails with an OutOfMemoryError exception. I see the same failure on trunk
> > as well.
> >
> >
> > On Mon, Jan 14, 2019 at 6:21 AM Elek, Marton  wrote:
> >
> >> Thanks Sunil to manage this release.
> >>
> >> +1 (non-binding)
> >>
> >> 1. built from the source (with clean local maven repo)
> >> 2. verified signatures + checksum
> >> 3. deployed 3 node cluster to Google Kubernetes Engine with generated
> >> k8s resources [1]
> >> 4. Executed basic HDFS commands
> >> 5. Executed basic yarn example jobs
> >>
> >> Marton
> >>
> >> [1]: FTR: resources:
> >> https://github.com/flokkr/k8s/tree/master/examples/hadoop , generator:
> >> https://github.com/elek/flekszible
> >>
> >>
> >> On 1/8/19 12:42 PM, Sunil G wrote:
> >> > Hi folks,
> >> >
> >> >
> >> > Thanks to all of you who helped in this release [1] and for helping to
> >> vote
> >> > for RC0. I have created second release candidate (RC1) for Apache
> Hadoop
> >> > 3.2.0.
> >> >
> >> >
> >> > Artifacts for this RC are available here:
> >> >
> >> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
> >> >
> >> >
> >> > RC tag in git is release-3.2.0-RC1.
> >> >
> >> >
> >> >
> >> > The maven artifacts are available via repository.apache.org at
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
> >> >
> >> >
> >> > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm
> >> PST.
> >> >
> >> >
> >> >
> >> > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> >> > additions
> >> >
> >> > are the highlights of this release.
> >> >
> >> > 1. Node Attributes Support in YARN
> >> >
> >> > 2. Hadoop Submarine project for running Deep Learning workloads on
> YARN
> >> >
> >> > 3. Support service upgrade via YARN Service API and CLI
> >> >
> >> > 4. HDFS Storage Policy Satisfier
> >> >
> >> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >> >
> >> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >> >
> >> > 7. Improvements in Router-based HDFS federation
> >> >
> >> >
> >> >
> >> > Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
> >> >
> >> > I have done few testing with my pseudo cluster. My +1 to start.
> >> >
> >> >
> >> >
> >> > Regards,
> >> >
> >> > Sunil
> >> >
> >> >
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >> >
> >> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> >> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> Resolved
> >> > ORDER BY fixVersion ASC
> >> >
> >>
> >> -
> >> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >>
> >>
>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Wangda Tan
+1 (Binding).

Deployed a local cluster from binary, and ran some sample sanity jobs.

Thanks Sunil for driving the release.

Best,
Wangda


On Mon, Jan 14, 2019 at 11:26 AM Virajith Jalaparti 
wrote:

> Thanks Sunil and others who have worked on the making this release happen!
>
> +1 (non-binding)
>
> - Built from source
> - Deployed a pseudo-distributed one node cluster
> - Ran basic wordcount, sort, pi jobs
> - Basic HDFS/WebHDFS commands
> - Ran all the ABFS driver tests against an ADLS Gen 2 account in EAST US
>
> Non-blockers (AFAICT): The following tests in ABFS (HADOOP-15407) fail:
> - For ACLs ({{ITestAzureBlobFilesystemAcl}}) -- However, I believe these
> have been fixed in trunk.
> - {{
> ITestAzureBlobFileSystemE2EScale#testWriteHeavyBytesToFileAcrossThreads}}
> fails with an OutOfMemoryError exception. I see the same failure on trunk
> as well.
>
>
> On Mon, Jan 14, 2019 at 6:21 AM Elek, Marton  wrote:
>
>> Thanks Sunil to manage this release.
>>
>> +1 (non-binding)
>>
>> 1. built from the source (with clean local maven repo)
>> 2. verified signatures + checksum
>> 3. deployed 3 node cluster to Google Kubernetes Engine with generated
>> k8s resources [1]
>> 4. Executed basic HDFS commands
>> 5. Executed basic yarn example jobs
>>
>> Marton
>>
>> [1]: FTR: resources:
>> https://github.com/flokkr/k8s/tree/master/examples/hadoop , generator:
>> https://github.com/elek/flekszible
>>
>>
>> On 1/8/19 12:42 PM, Sunil G wrote:
>> > Hi folks,
>> >
>> >
>> > Thanks to all of you who helped in this release [1] and for helping to
>> vote
>> > for RC0. I have created second release candidate (RC1) for Apache Hadoop
>> > 3.2.0.
>> >
>> >
>> > Artifacts for this RC are available here:
>> >
>> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
>> >
>> >
>> > RC tag in git is release-3.2.0-RC1.
>> >
>> >
>> >
>> > The maven artifacts are available via repository.apache.org at
>> >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
>> >
>> >
>> > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm
>> PST.
>> >
>> >
>> >
>> > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
>> > additions
>> >
>> > are the highlights of this release.
>> >
>> > 1. Node Attributes Support in YARN
>> >
>> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>> >
>> > 3. Support service upgrade via YARN Service API and CLI
>> >
>> > 4. HDFS Storage Policy Satisfier
>> >
>> > 5. Support Windows Azure Storage - Blob file system in Hadoop
>> >
>> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>> >
>> > 7. Improvements in Router-based HDFS federation
>> >
>> >
>> >
>> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>> >
>> > I have done few testing with my pseudo cluster. My +1 to start.
>> >
>> >
>> >
>> > Regards,
>> >
>> > Sunil
>> >
>> >
>> >
>> > [1]
>> >
>> >
>> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>> >
>> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
>> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
>> > ORDER BY fixVersion ASC
>> >
>>
>> -
>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>>
>>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Takanobu Asanuma
Thanks for driving the release, Sunil.

+1(non-binding)
 - verified checksums
 - built package from source code
 - started RBF(HDFS) cluster
 - verified Web UI (NN, RM, Router)
 - verified some operations of Router-based Federation
 - verified some operations of Erasure Coding

Regards,
- Takanobu

on 2019/01/08 20:42, "Sunil G" wrote:

Hi folks,


Thanks to all of you who helped in this release [1] and for helping to vote
for RC0. I have created second release candidate (RC1) for Apache Hadoop
3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/


RC tag in git is release-3.2.0-RC1.



The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1178/


This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.



3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]


https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC




Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-14 Thread Virajith Jalaparti
Thanks Sunil and others who have worked on the making this release happen!

+1 (non-binding)

- Built from source
- Deployed a pseudo-distributed one node cluster
- Ran basic wordcount, sort, pi jobs
- Basic HDFS/WebHDFS commands
- Ran all the ABFS driver tests against an ADLS Gen 2 account in EAST US

Non-blockers (AFAICT): The following tests in ABFS (HADOOP-15407) fail:
- For ACLs ({{ITestAzureBlobFilesystemAcl}}) -- However, I believe these
have been fixed in trunk.
- {{ITestAzureBlobFileSystemE2EScale#testWriteHeavyBytesToFileAcrossThreads}}
fails with an OutOfMemoryError exception. I see the same failure on trunk
as well.


On Mon, Jan 14, 2019 at 6:21 AM Elek, Marton  wrote:

> Thanks Sunil to manage this release.
>
> +1 (non-binding)
>
> 1. built from the source (with clean local maven repo)
> 2. verified signatures + checksum
> 3. deployed 3 node cluster to Google Kubernetes Engine with generated
> k8s resources [1]
> 4. Executed basic HDFS commands
> 5. Executed basic yarn example jobs
>
> Marton
>
> [1]: FTR: resources:
> https://github.com/flokkr/k8s/tree/master/examples/hadoop , generator:
> https://github.com/elek/flekszible
>
>
> On 1/8/19 12:42 PM, Sunil G wrote:
> > Hi folks,
> >
> >
> > Thanks to all of you who helped in this release [1] and for helping to
> vote
> > for RC0. I have created second release candidate (RC1) for Apache Hadoop
> > 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
> >
> >
> > RC tag in git is release-3.2.0-RC1.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1178/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm
> PST.
> >
> >
> >
> > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
> >
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-14 Thread Elek, Marton
Thanks Sunil to manage this release.

+1 (non-binding)

1. built from the source (with clean local maven repo)
2. verified signatures + checksum
3. deployed 3 node cluster to Google Kubernetes Engine with generated
k8s resources [1]
4. Executed basic HDFS commands
5. Executed basic yarn example jobs

Marton

[1]: FTR: resources:
https://github.com/flokkr/k8s/tree/master/examples/hadoop , generator:
https://github.com/elek/flekszible


On 1/8/19 12:42 PM, Sunil G wrote:
> Hi folks,
> 
> 
> Thanks to all of you who helped in this release [1] and for helping to vote
> for RC0. I have created second release candidate (RC1) for Apache Hadoop
> 3.2.0.
> 
> 
> Artifacts for this RC are available here:
> 
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
> 
> 
> RC tag in git is release-3.2.0-RC1.
> 
> 
> 
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
> 
> 
> This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.
> 
> 
> 
> 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
> 
> are the highlights of this release.
> 
> 1. Node Attributes Support in YARN
> 
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> 
> 3. Support service upgrade via YARN Service API and CLI
> 
> 4. HDFS Storage Policy Satisfier
> 
> 5. Support Windows Azure Storage - Blob file system in Hadoop
> 
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> 
> 7. Improvements in Router-based HDFS federation
> 
> 
> 
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> 
> I have done few testing with my pseudo cluster. My +1 to start.
> 
> 
> 
> Regards,
> 
> Sunil
> 
> 
> 
> [1]
> 
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> 
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
> 

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org



Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-12 Thread Sunil G
Thanks Weiwei and Zoltan,

For the content issue in documentation, all the latest features are added
in left panel. It seems whenever new features went in, it was not added as
a summary. I will wait for the voting to see any issues and make a call at
the last.

@zoltan, Thanks for bringing this issue to my attention. I think if i
deploy with new command what Marton has mentioned, we will not have this
problem. And for nexus, we are not doing any checksum validation. @Steve
Loughran   @Wangda Tan   @Vinod
Kumar Vavilapalli  could you please advise whether I
can re-run nexus bit again and create a new link w/o creating another RC.
Please advise.

Thanks
Sunil
On Fri, 11 Jan 2019 at 8:10 AM, Gabor Bota  wrote:

>   Thanks for the work Sunil!
>
>   +1 (non-binding)
>
>   checked out git tag release-3.2.0-RC1.
>   hadoop-aws integration (mvn verify) test run was successful on eu-west-1
> (a known issue is there, it's fixed in trunk)
>   built from source on Mac OS X 10.14.2, java version 8.0.181-oracle
>   deployed on a 3 node cluster
>   verified pi job, teragen, terasort and teravalidate
>
>   Regards,
>   Gabor Bota
>
> On Fri, Jan 11, 2019 at 1:11 PM Zoltan Haindrich  wrote:
>
>> Hello,
>>
>> I would like to note that it seems like 3.2.0-RC1 release misses some
>> source attachments (as all releases lately). David Phillips just commented
>> on that jira yesterday; and
>> I've just noticed that a release vote is already going onso I think
>> now is the best time to talk about this - because
>> https://issues.apache.org/jira/browse/HADOOP-15205
>> is open now for almost a year.
>>
>> This might be just a documentation related issue; but then the
>> HowToRelease doc should be updated.
>> Steve Loughran was able to publish the artifacts in question for 2.7.7 -
>> but releases before and after that are missing these source attachements.
>>
>> People working on downstream projects (or at least me) may find it harder
>> to work with hadoop packages; beacuse of the missing source attachments.
>>
>> example artifact which misses the sources:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1178/org/apache/hadoop/hadoop-mapreduce-client-core/3.2.0/
>>
>> cheers,
>> Zoltan
>>
>> On 1/8/19 12:42 PM, Sunil G wrote:
>> > Hi folks,
>> >
>> >
>> > Thanks to all of you who helped in this release [1] and for helping to
>> vote
>> > for RC0. I have created second release candidate (RC1) for Apache Hadoop
>> > 3.2.0.
>> >
>> >
>> > Artifacts for this RC are available here:
>> >
>> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
>> >
>> >
>> > RC tag in git is release-3.2.0-RC1.
>> >
>> >
>> >
>> > The maven artifacts are available via repository.apache.org at
>> >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
>> >
>> >
>> > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm
>> PST.
>> >
>> >
>> >
>> > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
>> > additions
>> >
>> > are the highlights of this release.
>> >
>> > 1. Node Attributes Support in YARN
>> >
>> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>> >
>> > 3. Support service upgrade via YARN Service API and CLI
>> >
>> > 4. HDFS Storage Policy Satisfier
>> >
>> > 5. Support Windows Azure Storage - Blob file system in Hadoop
>> >
>> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>> >
>> > 7. Improvements in Router-based HDFS federation
>> >
>> >
>> >
>> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>> >
>> > I have done few testing with my pseudo cluster. My +1 to start.
>> >
>> >
>> >
>> > Regards,
>> >
>> > Sunil
>> >
>> >
>> >
>> > [1]
>> >
>> >
>> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>> >
>> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
>> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
>> > ORDER BY fixVersion ASC
>> >
>>
>> -
>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>>
>>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-11 Thread Zoltan Haindrich

Hello,

I would like to note that it seems like 3.2.0-RC1 release misses some source attachments (as all releases lately). David Phillips just commented on that jira yesterday; and 
I've just noticed that a release vote is already going onso I think now is the best time to talk about this - because https://issues.apache.org/jira/browse/HADOOP-15205 
is open now for almost a year.


This might be just a documentation related issue; but then the HowToRelease doc 
should be updated.
Steve Loughran was able to publish the artifacts in question for 2.7.7 - but 
releases before and after that are missing these source attachements.

People working on downstream projects (or at least me) may find it harder to 
work with hadoop packages; beacuse of the missing source attachments.

example artifact which misses the sources:
https://repository.apache.org/content/repositories/orgapachehadoop-1178/org/apache/hadoop/hadoop-mapreduce-client-core/3.2.0/

cheers,
Zoltan

On 1/8/19 12:42 PM, Sunil G wrote:

Hi folks,


Thanks to all of you who helped in this release [1] and for helping to vote
for RC0. I have created second release candidate (RC1) for Apache Hadoop
3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/


RC tag in git is release-3.2.0-RC1.



The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1178/


This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.



3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC



-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org



Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-10 Thread Aaron Fabbri
Thanks Sunil and everyone who has worked on this release.

+1 from me.

- Verified checksums for tar file.
- Built from tar.gz.
- Ran through S3A and S3Guard integration tests (in AWS us-west 2).

This includes a yarn minicluster test but is mostly focused on s3a/s3guard.

Cheers,
Aaron


On Thu, Jan 10, 2019 at 2:32 PM Kuhu Shukla 
wrote:

> +1 (non-binding)
>
> - built from source on Mac
> - deployed on a pseudo distributed one node cluster
> - ran example jobs like sleep and wordcount.
>
> Thank you for all the work on this release.
> Regards,
> Kuhu
>
> On Thu, Jan 10, 2019 at 10:32 AM Craig.Condit 
> wrote:
>
> > +1 (non-binding)
> >
> > - built from source on CentOS 7.5
> > - deployed single node cluster
> > - ran several yarn jobs
> > - ran multiple docker jobs, including spark-on-docker
> >
> > On 1/8/19, 5:42 AM, "Sunil G"  wrote:
> >
> > Hi folks,
> >
> >
> > Thanks to all of you who helped in this release [1] and for helping
> to
> > vote
> > for RC0. I have created second release candidate (RC1) for Apache
> > Hadoop
> > 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
> >
> >
> > RC tag in git is release-3.2.0-RC1.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1178/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59
> pm
> > PST.
> >
> >
> >
> > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on
> YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the
> > release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> > (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status =
> Resolved
> > ORDER BY fixVersion ASC
> >
> >
> >
>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-10 Thread Kuhu Shukla
+1 (non-binding)

- built from source on Mac
- deployed on a pseudo distributed one node cluster
- ran example jobs like sleep and wordcount.

Thank you for all the work on this release.
Regards,
Kuhu

On Thu, Jan 10, 2019 at 10:32 AM Craig.Condit 
wrote:

> +1 (non-binding)
>
> - built from source on CentOS 7.5
> - deployed single node cluster
> - ran several yarn jobs
> - ran multiple docker jobs, including spark-on-docker
>
> On 1/8/19, 5:42 AM, "Sunil G"  wrote:
>
> Hi folks,
>
>
> Thanks to all of you who helped in this release [1] and for helping to
> vote
> for RC0. I have created second release candidate (RC1) for Apache
> Hadoop
> 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
>
>
> RC tag in git is release-3.2.0-RC1.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
>
>
> This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm
> PST.
>
>
>
> 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the
> release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
>
>


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-10 Thread Craig . Condit
+1 (non-binding)

- built from source on CentOS 7.5
- deployed single node cluster
- ran several yarn jobs
- ran multiple docker jobs, including spark-on-docker

On 1/8/19, 5:42 AM, "Sunil G"  wrote:

Hi folks,


Thanks to all of you who helped in this release [1] and for helping to vote
for RC0. I have created second release candidate (RC1) for Apache Hadoop
3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/


RC tag in git is release-3.2.0-RC1.



The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1178/


This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.



3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]


https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC




Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-09 Thread Wilfred Spiegelenburg
+1 (non binding)

- build from source on MacOSX 10.14.2, 1.8.0u181
- successful native build on Ubuntu 16.04.3
- confirmed the checksum and signature
- deployed a single node cluster  (openjdk 1.8u191 / centos 7.5)
- uploaded the MR framework
- configured YARN with the FS
- ran multiple MR jobs

> On 8 Jan 2019, at 22:42, Sunil G  wrote:
> 
> Hi folks,
> 
> 
> Thanks to all of you who helped in this release [1] and for helping to vote
> for RC0. I have created second release candidate (RC1) for Apache Hadoop
> 3.2.0.
> 
> 
> Artifacts for this RC are available here:
> 
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
> 
> 
> RC tag in git is release-3.2.0-RC1.
> 
> 
> 
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
> 
> 
> This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.
> 
> 
> 
> 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
> 
> are the highlights of this release.
> 
> 1. Node Attributes Support in YARN
> 
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> 
> 3. Support service upgrade via YARN Service API and CLI
> 
> 4. HDFS Storage Policy Satisfier
> 
> 5. Support Windows Azure Storage - Blob file system in Hadoop
> 
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> 
> 7. Improvements in Router-based HDFS federation
> 
> 
> 
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> 
> I have done few testing with my pseudo cluster. My +1 to start.
> 
> 
> 
> Regards,
> 
> Sunil
> 
> 
> 
> [1]
> 
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> 
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC


Wilfred Spiegelenburg | Software Engineer
cloudera.com 









Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-09 Thread Eric Badger
+1 (non-binding)

- Verified all hashes and checksums
- Built from source on RHEL 7
- Deployed a single node pseudo cluster
- Ran some example jobs
- Verified the Docker environment works with non-entrypoint mode

Eric

On Tue, Jan 8, 2019 at 5:42 AM Sunil G  wrote:

> Hi folks,
>
>
> Thanks to all of you who helped in this release [1] and for helping to vote
> for RC0. I have created second release candidate (RC1) for Apache Hadoop
> 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
>
>
> RC tag in git is release-3.2.0-RC1.
>
>
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1178/
>
>
> This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>


[VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-08 Thread Sunil G
Hi folks,


Thanks to all of you who helped in this release [1] and for helping to vote
for RC0. I have created second release candidate (RC1) for Apache Hadoop
3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/


RC tag in git is release-3.2.0-RC1.



The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1178/


This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.



3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC