Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-29 Thread Sunil G
Thanks @Eric Payne Due to another issue, we have to spin an RC1. In this case, could we revert the patch which caused this problem. Cancelling this RC0. Thanks to everyone who voted. I will spin RC1 as soon as possible. - Sunil On Fri, Nov 30, 2018 at 4:14 AM Eric Payne wrote: > The problem

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-29 Thread Eric Payne
The problem is not with preemption. The yarn-site.xml that I use for my pseudo-cluster includes a second xml: xi:include href=".../yarn-scheduler.xml" The property for yarn.resourcemanager.scheduler.monitor.enable = true is in this yarn-scheduler.xml. This value IS READ when then RM starts.

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-29 Thread Takanobu Asanuma
Thanks for driving the release, Sunil! +1(non-binding) - verified checksums - succeeded in building the package (CentOS7, Java 8u181) - started NN HA clusters - verified Web UI (NN, Router) - verified some operations of Erasure Coding - verified some operations of Router-based

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-29 Thread Zsolt Venczel
Thanks Sunil for preparing this release! +1 (non-binding) - Built from source at tag 3.2.0-rc0 (Ubuntu 16.04, JDK1.8.0_191) - Installed on a 4-node virtual cluster - Executed teragen/terasort/teravalidate with and without EC (using XOR-2-1-1024k policy) - Created two snapshots and performed

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-28 Thread Wilfred Spiegelenburg
+1 (non binding) - build from source on MacOSX 10.14.1, 1.8.0u181 - successful native build on Ubuntu 16.04.3 - confirmed the checksum and signature - deployed a single node cluster (jdk 1.8u191 / centos 7.5) Wilfred > On 23 Nov 2018, at 23:06, Sunil G wrote: > > Hi folks, > > > > Thanks

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-28 Thread Sunil G
Hi Eric, Thanks for helping in verifying the release. Post YARN-7370, preemption configs are refreshable. I tried to test by making some changes in capacity-scheduler.xml and invoking yarn rmadmin -refreshQueues. I can see the changes reflected as per logs after refresh. Could you please help to

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-28 Thread Chandni Singh
+1 (non-binding) - Build from source - Launched example sleeper service - Tested upgrade of sleeper service - Tested cancellation of upgrade - Tested express upgrade On Wed, Nov 28, 2018 at 8:22 AM Peter Bacsko wrote: > +1 (non-binding) > > - Built from source at tag 3.2.0-rc0 (Ubuntu 18.10,

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-28 Thread Eric Payne
Sunil, So, the basic symptoms are that if preemption is enabled on any queue, the preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all of the preemption-specific properties are set back to the default values. This was introduced in branch-3.1, so it is NOT new behavior for

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-28 Thread Eric Payne
Sunil, thanks for all of the hard work on this release. I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM. I am still investigating, but I thought I

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-28 Thread Peter Bacsko
+1 (non-binding) - Built from source at tag 3.2.0-rc0 (Ubuntu 18.10, JDK1.8.0_191) - Verified checksums of hadoop-3.2.0.tar.gz - Installed on a 3-node physical cluster - Ran teragen/terasort/teravalidate - Ran distributed shell a couple of times - Checked UIs (RM, NM, DN, JHS) Peter On Wed, Nov

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-28 Thread Jason Lowe
Thanks for driving this release, Sunil! +1 (binding) - Verified signatures and digests - Successfully performed a native build - Deployed a single-node cluster - Ran some sample jobs Jason On Fri, Nov 23, 2018 at 6:07 AM Sunil G wrote: > Hi folks, > > > > Thanks to all contributors who

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-27 Thread Akira Ajisaka
Thanks Sunil for driving this release. +1 (binding) - verified signatures and checksums - checked the documents including changes and release notes - built from source on CentOS 7.5, Java 1.8.0_191 - ran some MapReduce jobs in pseudo-distributed cluster - checked some web UIs (NN, DN, RM, NM)

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-27 Thread IƱigo Goiri
+1 (non-binding) - Deployed a cluster with 3 NNs, 3 RMs and 1 DN/NM on Azure - Tested the Active probe for the Load Balancer in front of the NNs and the RMs - Checked the NN, RBF, and RM Web UIs - Executed a wordcount, TeraGen, TeraSort and TeraValidate - Executed a YARN service with a

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-27 Thread Gabor Bota
Thanks for the work Sunil! +1 (non-binding) I've done the following: - checked out git tag release-3.2.0-RC0 - built from source on Mac OS X 10.14.1, java: 8.0.181-oracle - run hadoop-aws tests on AWS S3 against eu-west-1, with no unknown issues - deployed on a 3 node cluster

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-27 Thread Eric Badger
+1 (non-binding) - Verified all hashes and checksums - Built from source on macOS 10.14.1, Java 1.8.0u65 - Deployed a pseudo cluster - Ran some example jobs Eric On Tue, Nov 27, 2018 at 7:02 AM Ayush Saxena wrote: > Sunil, Thanks for driving this release!!! > > +1 (non-binding) > -Built from

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-27 Thread Ayush Saxena
Sunil, Thanks for driving this release!!! +1 (non-binding) -Built from source on Ubuntu 17.04 and JDK 8 -Ran basic Hdfs Commands -Ran basic EC Commands -Ran basic RBF commands -Browsed HDFS and RBF WEB UI -Ran TeraGen/TeraSort Regards Ayush > On 27-Nov-2018, at 4:06 PM, Kitti Nanasi wrote: >

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-27 Thread Kitti Nanasi
Thanks Sunil for the driving the release! +1 (non-binding) - checked out git tag release-3.2.0-RC0 - built from source on Mac OS X 10.13.4, java version 8.0.172-zulu - deployed on a 5 node cluster - ran terasort, teragen, teravalidate with success - executed basic hdfs, dfsadmin and ec commands

[VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-23 Thread Sunil G
Hi folks, Thanks to all contributors who helped in this release [1]. I have created first release candidate (RC0) for Apache Hadoop 3.2.0. Artifacts for this RC are available here: http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/ RC tag in git is release-3.2.0-RC0. The maven artifacts