[jira] [Created] (HDDS-308) SCM should identify a container with pending deletes using container reports
Lokesh Jain created HDDS-308: Summary: SCM should identify a container with pending deletes using container reports Key: HDDS-308 URL: https://issues.apache.org/jira/browse/HDDS-308 Project: Hadoop Distributed Data Store Issue Type: Bug Components: SCM Reporter: Lokesh Jain Assignee: Lokesh Jain Fix For: 0.2.1 SCM should fire an event when it finds using container report that a container's deleteTransactionID does not match SCM's deleteTransactionId. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-13775) Add a conf property to retry dns reverse lookups during datanode registration
Aniket Mokashi created HDFS-13775: - Summary: Add a conf property to retry dns reverse lookups during datanode registration Key: HDFS-13775 URL: https://issues.apache.org/jira/browse/HDFS-13775 Project: Hadoop HDFS Issue Type: Bug Components: namenode Reporter: Aniket Mokashi During DN registration, if the ip and the hostname in the DN InetAddress are the same (i.e. reverse name lookup falied; see [the javadoc|http://docs.oracle.com/javase/6/docs/api/java/net/InetAddress.html#getHostName%28%29]), the DN will be disallowed to register, where the DN InetAddress was obtained from the socket. It is a performance improvement added by HDFS-3990 to avoid unnecessary DNS resolutions. However, in an environment where DNS lookups can be flaky this can cause datanode registration to fail permanently. We propose to add a conf property to retry DNS reverse lookup during DN registration. Otherwise, clusters with flaky reverse DNS lookup will fail permanently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/544/ [Jul 30, 2018 1:15:58 PM] (msingh) HDDS-302. Fix javadoc and add implementation details in [Jul 30, 2018 3:48:42 PM] (nanda) HDDS-287. Add Close ContainerAction to Datanode#StateContext when the [Jul 30, 2018 5:25:07 PM] (cliang) HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc [Jul 30, 2018 5:55:19 PM] (bibinchundatt) YARN-8584. Several typos in Log Aggregation related classes. Contributed [Jul 30, 2018 10:00:29 PM] (szetszwo) HDDS-293. Reduce memory usage and object creation in KeyData. [Jul 31, 2018 12:44:18 AM] (jhung) YARN-7974. Allow updating application tracking url after registration. [Jul 31, 2018 11:57:51 AM] (nanda) HDDS-305. Datanode StateContext#addContainerActionIfAbsent will add -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 00m 00s) unit Specific tests: Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.cli.TestAclCLI hadoop.cli.TestAclCLIWithPosixAclInheritance hadoop.cli.TestCacheAdminCLI hadoop.cli.TestCryptoAdminCLI hadoop.cli.TestDeleteCLI hadoop.cli.TestErasureCodingCLI hadoop.cli.TestHDFSCLI hadoop.cli.TestXAttrCLI hadoop.fs.contract.hdfs.TestHDFSContractAppend hadoop.fs.contract.hdfs.TestHDFSContractConcat hadoop.fs.contract.hdfs.TestHDFSContractCreate hadoop.fs.contract.hdfs.TestHDFSContractDelete hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus hadoop.fs.contract.hdfs.TestHDFSContractMkdir hadoop.fs.contract.hdfs.TestHDFSContractOpen hadoop.fs.contract.hdfs.TestHDFSContractPathHandle hadoop.fs.contract.hdfs.TestHDFSContractRename hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory hadoop.fs.contract.hdfs.TestHDFSContractSeek hadoop.fs.contract.hdfs.TestHDFSContractSetTimes hadoop.fs.loadGenerator.TestLoadGenerator hadoop.fs.permission.TestStickyBit hadoop.fs.shell.TestHdfsTextCommand hadoop.fs.TestEnhancedByteBufferAccess hadoop.fs.TestFcHdfsCreateMkdir hadoop.fs.TestFcHdfsPermission hadoop.fs.TestFcHdfsSetUMask hadoop.fs.TestGlobPaths hadoop.fs.TestHDFSFileContextMainOperations hadoop.fs.TestHDFSMultipartUploader hadoop.fs.TestHdfsNativeCodeLoader hadoop.fs.TestResolveHdfsSymlink hadoop.fs.TestSWebHdfsFileContextMainOperations hadoop.fs.TestSymlinkHdfsDisable hadoop.fs.TestSymlinkHdfsFileContext hadoop.fs.TestSymlinkHdfsFileSystem hadoop.fs.TestUnbuffer hadoop.fs.TestUrlStreamHandler hadoop.fs.TestWebHdfsFileContextMainOperations hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot hadoop.fs.viewfs.TestViewFileSystemHdfs hadoop.fs.viewfs.TestViewFileSystemLinkFallback hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash hadoop.fs.viewfs.TestViewFileSystemWithAcls hadoop.fs.viewfs.TestViewFileSystemWithTruncate hadoop.fs.viewfs.TestViewFileSystemWithXAttrs hadoop.fs.viewfs.TestViewFsAtHdfsRoot hadoop.fs.viewfs.TestViewFsDefaultValue hadoop.fs.viewfs.TestViewFsFileStatusHdfs hadoop.fs.viewfs.TestViewFsHdfs hadoop.fs.viewfs.TestViewFsWithAcls hadoop.fs.viewfs.TestViewFsWithXAttrs hadoop.hdfs.client.impl.TestBlockReaderLocal hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy hadoop.hdfs.client.impl.TestBlockReaderRemote hadoop.hdfs.client.impl.TestClientBlockVerification hadoop.hdfs.crypto.TestHdfsCryptoStreams hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer hadoop.hdfs.qjournal.client.TestEpochsAreUnique hadoop.hdfs.qjournal.client.TestQJMWithFaults hadoop.hdfs.qjournal.client.TestQuorumJournalManager hadoop.hdfs.qjournal.server.TestJournal hadoop.hdfs.qjournal.server.TestJournalNode hadoop.hdfs.qjournal.server.TestJournalNodeMXBean hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.qjournal.server.TestJournalNodeSync hadoop.hdfs.qjournal.TestMiniJournalCluster hadoop.hdfs.qjournal.TestNNWithQJM hadoop.hdfs.qjournal.TestSecureNNWithQJM hadoop.hdfs.security.TestDelegationToken hadoop.hdfs.security.TestDelegationTokenForProxyUser hadoop.hdfs.security.token.block.TestBlockToken hadoop.hdfs.server.balancer.TestBalancer hadoop.hdfs.server.balancer.TestBalancerRPCDelay hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
[jira] [Created] (HDDS-307) docs link on ozone website is broken
Arpit Agarwal created HDDS-307: -- Summary: docs link on ozone website is broken Key: HDDS-307 URL: https://issues.apache.org/jira/browse/HDDS-307 Project: Hadoop Distributed Data Store Issue Type: Bug Components: document Reporter: Arpit Agarwal Fix For: 0.2.1 The docs link on _ozone.hadoop.apache.org_ is broken: [http://ozone.hadoop.apache.org/docs] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/ [Jul 30, 2018 2:53:44 AM] (sammi.chen) HADOOP-15607. AliyunOSS: fix duplicated partNumber issue in [Jul 30, 2018 9:18:04 AM] (sunilg) YARN-8591. [ATSv2] NPE while checking for entity acl in non-secure [Jul 30, 2018 10:20:04 AM] (brahma) HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to support minimum [Jul 30, 2018 1:15:58 PM] (msingh) HDDS-302. Fix javadoc and add implementation details in [Jul 30, 2018 3:48:42 PM] (nanda) HDDS-287. Add Close ContainerAction to Datanode#StateContext when the [Jul 30, 2018 5:25:07 PM] (cliang) HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc [Jul 30, 2018 5:55:19 PM] (bibinchundatt) YARN-8584. Several typos in Log Aggregation related classes. Contributed [Jul 30, 2018 10:00:29 PM] (szetszwo) HDDS-293. Reduce memory usage and object creation in KeyData. [Jul 31, 2018 12:44:18 AM] (jhung) YARN-7974. Allow updating application tracking url after registration. -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_libhdfs_threaded_hdfspp_test_shim_static Failed junit tests : hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestMRTimelineEventHandling cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/diff-compile-javac-root.txt [332K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/diff-checkstyle-root.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/whitespace-eol.txt [9.4M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-hdds_client.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-ozone_client.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-ozone_common.txt [28K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/branch-findbugs-hadoop-ozone_tools.txt [4.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/diff-javadoc-javadoc-root.txt [760K] CTEST: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/854/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
[jira] [Created] (HDDS-306) Add functionality to construct OpenContainerBlockMap on datanode restart
Shashikant Banerjee created HDDS-306: Summary: Add functionality to construct OpenContainerBlockMap on datanode restart Key: HDDS-306 URL: https://issues.apache.org/jira/browse/HDDS-306 Project: Hadoop Distributed Data Store Issue Type: Bug Components: Ozone Datanode Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee Fix For: 0.2.1 OpenContainerBlockMap contains list of blocks which are not committed on a Datanode. In case the Datanode restarts, we may need to reconstruct this map by reading block layout for each container and verifying it with the container DB. This is required to close the container on a single dataNode as well as for Ozone garbage collection -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
