[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455365#comment-16455365 ] Hudson commented on HDFS-12871: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14070 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14070/]) HDFS-12871. Ozone: Service Discovery: Adding REST server details in (omalley: rev 5e31b920f072af6d6a5556909c33fbb048bdf403) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/ksm/TestKeySpaceManagerRestInterface.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch, HDFS-12871-HDFS-7240.002.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451149#comment-16451149 ] Hudson commented on HDFS-12871: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14057/]) HDFS-12871. Ozone: Service Discovery: Adding REST server details in (xyao: rev fd4a4ce18e994b0b69d5d7e0b13867607920eece) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/ksm/TestKeySpaceManagerRestInterface.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch, HDFS-12871-HDFS-7240.002.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16320941#comment-16320941 ] Xiaoyu Yao commented on HDFS-12871: --- [~nandakumar131], you are right. I took back my previous comment. +1 for the v002 patch and I will commit it shortly. > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch, HDFS-12871-HDFS-7240.002.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16320871#comment-16320871 ] Nanda kumar commented on HDFS-12871: [~xyao], the {{build()}} in line 878/884 are for {{ServicePort}} (HTTP and HTTPS), and the one in line 886 is for {{ServiceInfo}} (dnServiceInfoBuilder). We need all the three build calls. > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch, HDFS-12871-HDFS-7240.002.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16320859#comment-16320859 ] Xiaoyu Yao commented on HDFS-12871: --- Thanks [~nandakumar131] for the update. One more comment on the latest patch, +1 after that is fixed. KeySapceManager.java Line 878/884: Can you remove the redundant".build());"as we only need to build it once at line 886? > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch, HDFS-12871-HDFS-7240.002.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16320268#comment-16320268 ] genericqa commented on HDFS-12871: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} docker {color} | {color:red} 6m 18s{color} | {color:red} Docker failed to build yetus/hadoop:d11161b. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HDFS-12871 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12905450/HDFS-12871-HDFS-7240.002.patch | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/22634/console | | Powered by | Apache Yetus 0.7.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch, HDFS-12871-HDFS-7240.002.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313863#comment-16313863 ] Xiaoyu Yao commented on HDFS-12871: --- Thanks [~nandakumar131] for the update. The 2nd patch looks good to me except that we need to support DN with both HTTP and HTTPS. We should remove the {else} part so that HTTP service port is always added and HTTPS service port is added optionally when the https port #> 0. > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313821#comment-16313821 ] genericqa commented on HDFS-12871: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} HDFS-7240 Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 47s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 43s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 16s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 56s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 17s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} HDFS-7240 passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 4s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}174m 39s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 25s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}236m 28s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.ozone.web.client.TestKeysRatis | | | hadoop.ozone.ozShell.TestOzoneShell | | | hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy | | | hadoop.hdfs.TestLeaseRecovery2 | | | hadoop.hdfs.TestApplyingStoragePolicy | | | hadoop.hdfs.TestDFSStripedInputStream | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180 | | | hadoop.hdfs.TestDFSStartupVersions | | | hadoop.hdfs.TestDataTransferKeepalive | | | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 | | | hadoop.hdfs.TestDisableConnCache | | | hadoop.ozone.client.rpc.TestOzoneRpcClient | | | hadoop.ozone.container.replication.TestContainerReplicationManager | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | | | hadoop.hdfs.server.namenode.TestReencryptionWithKMS | | | hadoop.hdfs.TestErasureCodingMultipleRacks | | | hadoop.ozone.ksm.TestKeySpaceManager | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 | | | hadoop.hdfs.qjournal.server.TestJournalNodeSync | | | hadoop.ozone.scm.TestSCMCli | | | hadoop.ozone.web.client.TestKeys | | | hadoop.hdfs.TestDistributedFileSystemWithECFileWithRandomECPolicy | | | hadoop.hdfs.TestFileChecksum | \\ \\ || Subsystem ||
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16313392#comment-16313392 ] Nanda kumar commented on HDFS-12871: Thanks [~xyao] for the review and suggestions. bq. should we return HTTPS port only if HTTPS is enabled by Netty HTTP server? Good idea, have address this in patch v001. bq. For large cluster with thousands of DNs, do we still want to add all of them to the getServiceList servlet result? HDFS-12991 is created to handle this. > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar > Attachments: HDFS-12871-HDFS-7240.000.patch, > HDFS-12871-HDFS-7240.001.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311950#comment-16311950 ] Xiaoyu Yao commented on HDFS-12871: --- Thanks [~nandakumar131] for reporting the issue and posting the patch. It looks good to me. I just have two questions: KeySpaceManager.java Line 877-880: should we return HTTPS port only if HTTPS is enabled by Netty HTTP server? i.e., infoSecurePort is set with a valid value by DN? For large cluster with thousands of DNs, do we still want to add all of them to the getServiceList servlet result? Could this hit the max size of HTTP response header size (e.g., 8K) easily? > Ozone: Service Discovery: Adding REST server details in ServiceList > --- > > Key: HDFS-12871 > URL: https://issues.apache.org/jira/browse/HDFS-12871 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Nanda kumar > Attachments: HDFS-12871-HDFS-7240.000.patch > > > The datanode (REST server) details has to be added as part of > ServiceDiscovery {{getServiceList}} call. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12871) Ozone: Service Discovery: Adding REST server details in ServiceList
[ https://issues.apache.org/jira/browse/HDFS-12871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311902#comment-16311902 ] genericqa commented on HDFS-12871: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 31s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} HDFS-7240 Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 55s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 42s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 8s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 0s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 21s{color} | {color:green} HDFS-7240 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s{color} | {color:green} HDFS-7240 passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 9s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 9s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 42s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 29s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 16s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 19s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 87m 22s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestReadStripedFileWithDecodingCorruptData | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 | | | hadoop.hdfs.TestDFSClientRetries | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure190 | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d11161b | | JIRA Issue | HDFS-12871 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12904642/HDFS-12871-HDFS-7240.000.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 539af2733a84 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | HDFS-7240 / 4511825 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/22557/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/22557/testReport/ | | Max. process+thread count | 2443 (vs. ulimit of 5000) | | modules | C: