[jira] [Created] (HDDS-2152) Ozone client fails with OOM while writing a large (~300MB) key.

2019-09-18 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDDS-2152: --- Summary: Ozone client fails with OOM while writing a large (~300MB) key. Key: HDDS-2152 URL: https://issues.apache.org/jira/browse/HDDS-2152 Project: Hadoop

[jira] [Created] (HDDS-2151) Ozone client prints the entire request payload in DEBUG level.

2019-09-18 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDDS-2151: --- Summary: Ozone client prints the entire request payload in DEBUG level. Key: HDDS-2151 URL: https://issues.apache.org/jira/browse/HDDS-2151 Project: Hadoop

[jira] [Resolved] (HDDS-2021) Upgrade Guava library to v26 in hdds project

2019-09-18 Thread Dinesh Chitlangia (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Chitlangia resolved HDDS-2021. - Resolution: Not A Problem > Upgrade Guava library to v26 in hdds project >

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2019-09-18 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/449/ [Sep 18, 2019 5:31:33 PM] (ekrogen) HDFS-14569. Result of crypto -listZones is not formatted properly. [Sep 18, 2019 9:51:21 PM] (kihwal) HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky.

[jira] [Created] (HDDS-2150) Update dependency versions to avoid security vulnerabilities

2019-09-18 Thread Hanisha Koneru (Jira)
Hanisha Koneru created HDDS-2150: Summary: Update dependency versions to avoid security vulnerabilities Key: HDDS-2150 URL: https://issues.apache.org/jira/browse/HDDS-2150 Project: Hadoop Distributed

[jira] [Created] (HDDS-2149) Replace findbugs with spotbugs

2019-09-18 Thread Dinesh Chitlangia (Jira)
Dinesh Chitlangia created HDDS-2149: --- Summary: Replace findbugs with spotbugs Key: HDDS-2149 URL: https://issues.apache.org/jira/browse/HDDS-2149 Project: Hadoop Distributed Data Store

Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread 俊平堵
+1. Thanks, Junping Elek, Marton 于2019年9月17日周二 下午5:48写道: > > > TLDR; I propose to move Ozone related code out from Hadoop trunk and > store it in a separated *Hadoop* git repository apache/hadoop-ozone.git > > > > > When Ozone was adopted as a new Hadoop subproject it was proposed[1] to > be

[jira] [Resolved] (HDDS-2121) Create a shaded ozone filesystem (client) jar

2019-09-18 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham resolved HDDS-2121. -- Fix Version/s: 0.5.0 Resolution: Fixed > Create a shaded ozone filesystem

[jira] [Resolved] (HDFS-14846) libhdfs tests are failing on trunk due to jni usage bugs

2019-09-18 Thread Sahil Takiar (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar resolved HDFS-14846. - Fix Version/s: 3.3.0 Resolution: Fixed > libhdfs tests are failing on trunk due to jni

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Dinesh Chitlangia
+1 (non-binding) - built from the source on centos7 - verified signatures & sha512 checksums - verified basic HDFS operations with CLI As Steve already mentioned, we must mention about Guava. Thanks for organizing the release Rohit. Thanks, Dinesh On Wed, Sep 18, 2019 at 2:20 PM Steve

[jira] [Created] (HDDS-2148) Remove redundant code in CreateBucketHandler.java

2019-09-18 Thread Dinesh Chitlangia (Jira)
Dinesh Chitlangia created HDDS-2148: --- Summary: Remove redundant code in CreateBucketHandler.java Key: HDDS-2148 URL: https://issues.apache.org/jira/browse/HDDS-2148 Project: Hadoop Distributed Data

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Steve Loughran
On Wed, Sep 18, 2019 at 6:04 PM Rohith Sharma K S wrote: > Thanks Steve for detailed verification. Inline comment > > On Wed, 18 Sep 2019 at 20:34, Steve Loughran > wrote: > > > > > > > +1 binding. > > > > > > One caveat: warn people that guava is now at 27.0 -and that if you run > > > with an

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Elek, Marton
+1 (binding) Thanks Rohith the work with the release. * built from the source (archlinux) * verified signatures * verified sha512 checksums * started a docker-based pseudo cluster * tested basic HDFS operations with CLI * Checked if the sources are uploaded to the maven staging repo

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Masatake Iwasaki
+0 * built from source taball. * launched 3-nodes cluster. * ran some examplr MR jobs. * ran some CLI file operations. Same as 3.1.3 RC0, verbose INFO message is emitted on data transfer operations.::   $ bin/hadoop fs -put README.txt '/c;d/'   2019-09-18 17:41:24,977 INFO

Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread Arpit Agarwal
+1 > On Sep 17, 2019, at 2:49 AM, Elek, Marton wrote: > > > > TLDR; I propose to move Ozone related code out from Hadoop trunk and store it > in a separated *Hadoop* git repository apache/hadoop-ozone.git > > > > > When Ozone was adopted as a new Hadoop subproject it was proposed[1] to

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Rohith Sharma K S
Thanks Steve for detailed verification. Inline comment On Wed, 18 Sep 2019 at 20:34, Steve Loughran wrote: > > > > +1 binding. > > > > One caveat: warn people that guava is now at 27.0 -and that if you run > > with an older version of Guava things will inevitably break. > >: Could you

[jira] [Created] (HDDS-2147) Include dumpstream in test report

2019-09-18 Thread Doroszlai, Attila (Jira)
Doroszlai, Attila created HDDS-2147: --- Summary: Include dumpstream in test report Key: HDDS-2147 URL: https://issues.apache.org/jira/browse/HDDS-2147 Project: Hadoop Distributed Data Store

[jira] [Created] (HDDS-2146) Optimize block write path performance by reducing no of watchForCommit calls

2019-09-18 Thread Shashikant Banerjee (Jira)
Shashikant Banerjee created HDDS-2146: - Summary: Optimize block write path performance by reducing no of watchForCommit calls Key: HDDS-2146 URL: https://issues.apache.org/jira/browse/HDDS-2146

[jira] [Created] (HDDS-2145) Optimize client read path by reading multiple chunks along with block info in a single rpc call.

2019-09-18 Thread Shashikant Banerjee (Jira)
Shashikant Banerjee created HDDS-2145: - Summary: Optimize client read path by reading multiple chunks along with block info in a single rpc call. Key: HDDS-2145 URL:

Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread Elek, Marton
> one thing to consider here as you are giving up your ability to make > changes in hadoop-* modules, including hadoop-common, and their > dependencies, in sync with your own code. That goes for filesystem contract > tests. > > are you happy with that? Yes. I think we can live with it.

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Steve Loughran
> > +1 binding. > > One caveat: warn people that guava is now at 27.0 -and that if you run > with an older version of Guava things will inevitably break. > > > steps to validate > == > > * downloaded src and binary artifacts > * after import of KEYS and trusting Rohith's key,

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Ayush Saxena
Thanks Rohith for driving the release. +1 (non binding) -Built from source on Ubuntu-18.04 -Successful Native build. -Verified basic HDFS Commands. -Verified basic Erasure Coding Commands. -Verified basic RBF commands. -Browsed HDFS UI. Thanks -Ayush On Wed, 18 Sep 2019 at 15:41, Weiwei Yang

[jira] [Created] (HDFS-14854) Create improved decommission monitor implementation

2019-09-18 Thread Stephen O'Donnell (Jira)
Stephen O'Donnell created HDFS-14854: Summary: Create improved decommission monitor implementation Key: HDFS-14854 URL: https://issues.apache.org/jira/browse/HDFS-14854 Project: Hadoop HDFS

[jira] [Created] (HDFS-14853) NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is deleted

2019-09-18 Thread Ranith Sardar (Jira)
Ranith Sardar created HDFS-14853: Summary: NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is deleted Key: HDFS-14853 URL: https://issues.apache.org/jira/browse/HDFS-14853

[jira] [Created] (HDFS-14852) Remove of LowRedundancyBlocks do NOT remove the block from all queues

2019-09-18 Thread Fei Hui (Jira)
Fei Hui created HDFS-14852: -- Summary: Remove of LowRedundancyBlocks do NOT remove the block from all queues Key: HDFS-14852 URL: https://issues.apache.org/jira/browse/HDFS-14852 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 3.2.1 - RC0

2019-09-18 Thread Weiwei Yang
+1 (binding) Downloaded tarball, setup a pseudo cluster manually Verified basic HDFS operations, copy/view files Verified basic YARN operations, run sample DS jobs Verified basic YARN restful APIs, e.g cluster/nodes info etc Set and verified YARN node-attributes, including CLI Thanks Weiwei On

Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree

2019-09-18 Thread Steve Loughran
one thing to consider here as you are giving up your ability to make changes in hadoop-* modules, including hadoop-common, and their dependencies, in sync with your own code. That goes for filesystem contract tests. are you happy with that? On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton wrote:

Re: [VOTE] Release Hadoop-3.1.3-RC0

2019-09-18 Thread zhankun tang
Hi Masatake, My bad. I read it wrong. Yeah. You're right. The HDFS-14759 can be backported to both Branch-3.2 and 3.1. Let's see if there's any other blocker/critical issue come up, otherwise, personally I don't prefer to run another RC1 and vote for this minor change. BR, Zhankun On Wed, 18

Re: [VOTE] Release Hadoop-3.1.3-RC0

2019-09-18 Thread Masatake Iwasaki
Hi Zhankun, > Can you please help to provide a branch-3.1 patch for HDFS-14759? Or we can > move it to the next release of branch-3.1 since the noisy info is not a > blocker issue to me. Does that make sense? I tried to cherry-picking the HDFS-14759 on my side. Since there was no conflict, I

Re: [VOTE] Release Hadoop-3.1.3-RC0

2019-09-18 Thread Zhankun Tang
Hi Masatake, Thanks for helping to verify! I checked that branch-3.2 has the HDFS-14759 committed already. Release-3.2.1-RC0 should have no such issue. For branch-3.1, cherry-pick the same commit has conflicts. I'm confirming if we can fix it or there's a feasible plan to backport the whole

Re: [VOTE] Release Hadoop-3.1.3-RC0

2019-09-18 Thread Masatake Iwasaki
Thanks for putting this up, Zhankun Tang. While I was testing the RC0 wich CLI, noisy INFO message was emitted on every data transfer operation.::   2019-09-17 16:00:42,942 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false The