[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580621#comment-16580621 ] Hudson commented on HADOOP-14212: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14773 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14773/]) HADOOP-14212. Addendum patch: Expose SecurityEnabled boolean field in (weichiu: rev bdd0e01ed9d8ecd064a90fd99ef7652b90c025bd) * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerMXBean.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManagerMXBean.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManagerMXBean.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManagerMXBean.java > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Fix For: 3.2.0 > > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580613#comment-16580613 ] Weiwei Yang commented on HADOOP-14212: -- Thanks [~jojochuang] for taking quick action to get this fixed, appreciate that! > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Fix For: 3.2.0 > > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580608#comment-16580608 ] Wei-Chiu Chuang commented on HADOOP-14212: -- I added the missing files and verified it compiles after the addendum patch. Thanks again [~cheersyang] for catching the issue quickly > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Fix For: 3.2.0 > > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580603#comment-16580603 ] Wei-Chiu Chuang commented on HADOOP-14212: -- [~cheersyang] It's my fault. I'm deeply sorry about that. Will get this addressed asap. Thanks > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Fix For: 3.2.0 > > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580597#comment-16580597 ] Weiwei Yang commented on HADOOP-14212: -- Hi [~jojochuang] It looks like you some files are missing in commit 363bd16e314490e31758cc82de584e75cd8357cc, e.g NodeManagerMXBean.java, ResourceManager.java, causing the build compilation to fail. Could you please fix this? Thanks > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Fix For: 3.2.0 > > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580572#comment-16580572 ] Hudson commented on HADOOP-14212: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #14771 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14771/]) HADOOP-14212. Expose SecurityEnabled boolean field in JMX for other (weichiu: rev 363bd16e314490e31758cc82de584e75cd8357cc) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeMXBean.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMXBean.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNodeInfoMXBean.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNode.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580222#comment-16580222 ] Wei-Chiu Chuang commented on HADOOP-14212: -- +1 Thanks for working on the patch, [~adam.antal] The patch looks good to me. For HDFS, I am not sure how useful it is to add isSecurityEnabled for roles other than NameNode, because supposedly every role in the cluster must be secured if the NameNode is secured. I guess it doesn't hurt to add though. But it could be useful for YARN roles. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566921#comment-16566921 ] Adam Antal commented on HADOOP-14212: - As I see the InterfaceAudience annotation is changed in HDFS-11217 - so the patch is good as for the annotation. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564079#comment-16564079 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 34s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 24s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 3s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 7s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 3s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 3s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m 7s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 59s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 46s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}288m 5s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | | | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933762/HADOOP-14212.009.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 228701117dc7 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool |
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563633#comment-16563633 ] Adam Antal commented on HADOOP-14212: - Thank you for the suggestions, [~gabor.bota]. I uploaded patch v9 with these items, and also ran an additional checkstyle. One minor side question of mine is: shouldn't the {{@InterfaceAudience.Private}} annotation be used since end users shouldn't be implementing those interfaces - they should use the API instead. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563352#comment-16563352 ] Gabor Bota commented on HADOOP-14212: - Thanks for working on this [~adam.antal]! I've noticed the following while I did a review: * In {{SecondaryNameNodeInfoMXBean}} you removed {{public}} modifiers. It's generally not encouraged to do that until you have some real reason to do that (like increasing visibility for testing purposes). I'd recommend just leaving it as it is. * In {{TestDataNodeMXBean}} you've added a new test, which is very good. But for {{TestDataNodeMXBean #testDataNodeMXBean}} changing the try-catch to try-with-resources could cause backporting problems in the future, so it would be wise to change just what you have to in this case. * In {{NodeManager#serviceInit) there's a whitespace before the parentheses - it's just a minor flaw, but still would be better not to have it * In {{NodeManagerMXBean}} and {{ResourceManagerMXBean}} interfaces you should add {{@InterfaceAudience.Public}} and {{@InterfaceStability.Evolving}}. See eg. [Interface Taxonomy|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/InterfaceClassification.html] * For the class {{ResourceManager}} line 144 is longer than just 80 chars > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562485#comment-16562485 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 35s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 15s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 34m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 39m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 17m 10s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 54s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 16s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 32m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 32m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 10s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 5s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 54s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m 29s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 70m 42s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 48s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}350m 14s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | | | hadoop.hdfs.client.impl.TestBlockReaderLocal | | | hadoop.hdfs.server.datanode.TestDataNodeUUID | | | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933597/HADOOP-14212.008.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux d90953e6abea 3.13.0
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561992#comment-16561992 ] Adam Antal commented on HADOOP-14212: - Thanks for checking the patch [~ajayydv]. I added a test for security enabled testing of NameNodeStatusMXBean in patch v8. I have no new idea about NodeManager and ResourceManager. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch, HADOOP-14212.008.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560365#comment-16560365 ] Ajay Kumar commented on HADOOP-14212: - [~adam.antal] thanks for updating the patch. {quote} I used SaslDataTransferTestCase as base class for getting kerberos configuration, but this class is inaccessible from hadoop-yarn. Would you please advise how to surpass this?{quote} We can update TestSecureNameNode to include test case for NameNode. This class already extends SaslDataTransferTestCase. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556270#comment-16556270 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 2s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 31s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 44s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 8s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 30m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 17s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 2s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 8s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m 28s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 16s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 59s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}328m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.TestNameNodeMXBean | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933061/HADOOP-14212.007.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux f16ab6a3634f 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15 17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555835#comment-16555835 ] Adam Antal commented on HADOOP-14212: - Fixed a test related to my modifications (patch v7). > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch, > HADOOP-14212.007.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554795#comment-16554795 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 32m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 38m 42s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 16m 41s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 14s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 30m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 58s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 3s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 42s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m 23s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 70m 1s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 46s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}346m 40s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | | | hadoop.hdfs.server.datanode.TestDataNodeMXBean | | | hadoop.hdfs.TestCrcCorruption | | | hadoop.hdfs.server.namenode.TestFSImageWithXAttr | | | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart | | | hadoop.yarn.server.resourcemanager.scheduler.fair.policies.TestDominantResourceFairnessPolicy | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12932899/HADOOP-14212.006.patch | | Optional
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554358#comment-16554358 ] Adam Antal commented on HADOOP-14212: - Thank you for the suggestions [~ajayydv], I worked accordingly: removed the redundant public keywords from the MXBeans, although in TestDataNodeMXBean.java the contents of testDataNodeMXBean test is in a try-with-resources block from line 60 to 117, that is why the finally clause got removed - so I see no point in not removing it. I added a test for kerberos authentication to TestDataNodeMXBean, could you please check if my modifications are appropriate? Also another question has arisen while I attempted to make tests for ResourceManager- and NodeManagerMXBean. I used SaslDataTransferTestCase as base class for getting kerberos configuration, but this class is inaccessible from hadoop-yarn. Would you please advise how to surpass this? (Moving SaslDataTransferTestCase to hadoop-common is not a good idea since it uses DFSConfigKeys from hadoop-hdfs, but introducting a new dependency has also some other caveats.) A simpler answer could be just to copy the code to make the tests for those two MXBeans in yarn, or don't write test for them. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553272#comment-16553272 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 48s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 22s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 2s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 29s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 3s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 16s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m 12s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 48s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 53s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}296m 35s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.client.impl.TestBlockReaderLocal | | | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12932707/HADOOP-14212.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux d75a07909d5a 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality |
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552996#comment-16552996 ] Ajay Kumar commented on HADOOP-14212: - [~adam.antal] thanks for working on this. * TestDataNodeMXBean: L114-117 removal of finally block is not required. * Shall we test case when security is enabled as well? (In current test cases we are getting default value i.e false) Checkstyle NITs: ResourceManagerMXBean,SecondaryNameNodeInfoMXBean, : public modifier is redundant in function signatures. DataNodeMxBean: L155 public modifier is redundant. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch, HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552846#comment-16552846 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color} | {color:red} HADOOP-14212 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12932700/HADOOP-14212.005.patch | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14930/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552833#comment-16552833 ] Adam Antal commented on HADOOP-14212: - Reuploaded the same patch to trigger jenkins. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch, > HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545199#comment-16545199 ] Adam Antal commented on HADOOP-14212: - I couldn't recreate the failed test in my computer, so couldn't come up with new idea about the metrics. Maybe retry jenkins? > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544942#comment-16544942 ] Adam Antal commented on HADOOP-14212: - Yes, you're right - I'm looking into it. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543761#comment-16543761 ] Steve Loughran commented on HADOOP-14212: - Not sure what is up there; need to see what other jenkins builds are up to. Some of those stacks look related to metrics, which implies that the changes may be breaking assertions > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543406#comment-16543406 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 50s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 42s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 31s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 2s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 6s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 2s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 11s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 20s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 71m 42s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 51s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}317m 37s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeMetrics | | | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA | | | hadoop.yarn.server.nodemanager.containermanager.TestContainerManager | | | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12931471/HADOOP-14212.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux a25f99edce84 3.13.
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542954#comment-16542954 ] Adam Antal commented on HADOOP-14212: - Thanks you Steve, hope its all good. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542891#comment-16542891 ] Steve Loughran commented on HADOOP-14212: - LGTM, +1 attached patch 005 for jenkins, which is the one I'll be applying: reorder imports and add a missing EOF marker nothing significant, but submitting for due diligence > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542095#comment-16542095 ] genericqa commented on HADOOP-14212: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 50s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 33m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 53s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 23s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 31m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 31m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 9s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 2s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 74m 56s{color} | {color:green} hadoop-hdfs in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 17m 50s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 64m 10s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 48s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}305m 31s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HADOOP-14212 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12931328/HADOOP-14212.004.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux cef7a4316183 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / b37074b | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_171 | | f
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541679#comment-16541679 ] Adam Antal commented on HADOOP-14212: - Thank you for looking into it, I used try-with-resources in all the tests (uploaded patch). Waiting for jenkins now. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch, HADOOP-14212.004.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540241#comment-16540241 ] Steve Loughran commented on HADOOP-14212: - I like publishing info as metrics, we use it for S3A perf testing. One concern of people running things in production are cost of metric creation and stability. I don't see us removing this. while jenkins does its testing, quick comment on the tests. in {{testResourceManagerMXBean}} you should close() the RM afterwards; using a try-with-resources should do that. thanks > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540214#comment-16540214 ] Adam Antal commented on HADOOP-14212: - MXBeans added for ResourceManager and NodeManager, added test too. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch, > HADOOP-14212.003.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540076#comment-16540076 ] Adam Antal commented on HADOOP-14212: - Patch for DN with tests uploaded. Modified test passed in my computer. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[ https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540003#comment-16540003 ] Adam Antal commented on HADOOP-14212: - Added patch for SNN. Unit testing is not applicable (no testcases for SecondaryNameNodeInfoMXBean) Continue working on other services. > Expose SecurityEnabled boolean field in JMX for other services besides > NameNode > --- > > Key: HADOOP-14212 > URL: https://issues.apache.org/jira/browse/HADOOP-14212 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Burgemeestre >Assignee: Adam Antal >Priority: Minor > Labels: newbie, security > Attachments: HADOOP-14212.001.patch > > > The following commit > https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12 > introduced a "SecurityEnabled" field in the JMX output for the NameNode. I > believe it would be nice to add this same change to the JMX output of other > services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc. > So that it can be queried whether Security is enabled in all JMX resources. > The reason I am suggesting this feature / improvement is that I think it > would provide a clean way to check whether your cluster is completely > Kerberized or not. I don't think there is an easy/clean way to do this now, > other than checking the logs, checking ports etc.? > The file where the change was made is > hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java > has the following function now: > {code:java} > @Override // NameNodeStatusMXBean > public boolean isSecurityEnabled() { > return UserGroupInformation.isSecurityEnabled(); > } > {code} > I would be happy to develop a patch if it seems useful by others as well? > This is a snippet from the JMX output from the NameNode in case security is > not enabled: > {code} > { > "name" : "Hadoop:service=NameNode,name=NameNodeStatus", > "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode", > "NNRole" : "NameNode", > "HostAndPort" : "node001.cm.cluster:8020", > "SecurityEnabled" : false, > "LastHATransitionTime" : 0, > "State" : "standby" > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org