[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580621#comment-16580621
]
Hudson commented on HADOOP-14212:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14773 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14773/])
HADOOP-14212. Addendum patch: Expose SecurityEnabled boolean field in (weichiu:
rev bdd0e01ed9d8ecd064a90fd99ef7652b90c025bd)
* (add)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerMXBean.java
* (add)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManagerMXBean.java
* (add)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManagerMXBean.java
* (add)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManagerMXBean.java
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Fix For: 3.2.0
>
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580613#comment-16580613
]
Weiwei Yang commented on HADOOP-14212:
--
Thanks [~jojochuang] for taking quick action to get this fixed, appreciate that!
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Fix For: 3.2.0
>
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580608#comment-16580608
]
Wei-Chiu Chuang commented on HADOOP-14212:
--
I added the missing files and verified it compiles after the addendum patch.
Thanks again [~cheersyang] for catching the issue quickly
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Fix For: 3.2.0
>
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580603#comment-16580603
]
Wei-Chiu Chuang commented on HADOOP-14212:
--
[~cheersyang] It's my fault. I'm deeply sorry about that. Will get this
addressed asap. Thanks
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Fix For: 3.2.0
>
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580597#comment-16580597
]
Weiwei Yang commented on HADOOP-14212:
--
Hi [~jojochuang]
It looks like you some files are missing in commit
363bd16e314490e31758cc82de584e75cd8357cc, e.g NodeManagerMXBean.java,
ResourceManager.java, causing the build compilation to fail. Could you please
fix this?
Thanks
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Fix For: 3.2.0
>
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580572#comment-16580572
]
Hudson commented on HADOOP-14212:
-
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #14771 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14771/])
HADOOP-14212. Expose SecurityEnabled boolean field in JMX for other (weichiu:
rev 363bd16e314490e31758cc82de584e75cd8357cc)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeMXBean.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMXBean.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNodeInfoMXBean.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNode.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580222#comment-16580222
]
Wei-Chiu Chuang commented on HADOOP-14212:
--
+1 Thanks for working on the patch, [~adam.antal] The patch looks good to me.
For HDFS, I am not sure how useful it is to add isSecurityEnabled for roles
other than NameNode, because supposedly every role in the cluster must be
secured if the NameNode is secured. I guess it doesn't hurt to add though.
But it could be useful for YARN roles.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566921#comment-16566921
]
Adam Antal commented on HADOOP-14212:
-
As I see the InterfaceAudience annotation is changed in HDFS-11217 - so the
patch is good as for the annotation.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564079#comment-16564079
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 24s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 3s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
7s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 59s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}288m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
| | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12933762/HADOOP-14212.009.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 228701117dc7 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563633#comment-16563633
]
Adam Antal commented on HADOOP-14212:
-
Thank you for the suggestions, [~gabor.bota]. I uploaded patch v9 with these
items, and also ran an additional checkstyle.
One minor side question of mine is: shouldn't the
{{@InterfaceAudience.Private}} annotation be used since end users shouldn't be
implementing those interfaces - they should use the API instead.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch, HADOOP-14212.009.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563352#comment-16563352
]
Gabor Bota commented on HADOOP-14212:
-
Thanks for working on this [~adam.antal]!
I've noticed the following while I did a review:
* In {{SecondaryNameNodeInfoMXBean}} you removed {{public}} modifiers. It's
generally not encouraged to do that until you have some real reason to do that
(like increasing visibility for testing purposes). I'd recommend just leaving
it as it is.
* In {{TestDataNodeMXBean}} you've added a new test, which is very good. But
for {{TestDataNodeMXBean #testDataNodeMXBean}} changing the try-catch to
try-with-resources could cause backporting problems in the future, so it would
be wise to change just what you have to in this case.
* In {{NodeManager#serviceInit) there's a whitespace before the parentheses -
it's just a minor flaw, but still would be better not to have it
* In {{NodeManagerMXBean}} and {{ResourceManagerMXBean}} interfaces you should
add {{@InterfaceAudience.Public}} and {{@InterfaceStability.Evolving}}. See eg.
[Interface
Taxonomy|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/InterfaceClassification.html]
* For the class {{ResourceManager}} line 144 is longer than just 80 chars
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562485#comment-16562485
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 34m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 39m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 10s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
16s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 32m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 32m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 54s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
29s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 70m
42s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}350m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.client.impl.TestBlockReaderLocal |
| | hadoop.hdfs.server.datanode.TestDataNodeUUID |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12933597/HADOOP-14212.008.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux d90953e6abea 3.13.0-144-generic
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16561992#comment-16561992
]
Adam Antal commented on HADOOP-14212:
-
Thanks for checking the patch [~ajayydv].
I added a test for security enabled testing of NameNodeStatusMXBean in patch
v8.
I have no new idea about NodeManager and ResourceManager.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch, HADOOP-14212.008.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560365#comment-16560365
]
Ajay Kumar commented on HADOOP-14212:
-
[~adam.antal] thanks for updating the patch.
{quote} I used SaslDataTransferTestCase as base class for getting kerberos
configuration, but this class is inaccessible from hadoop-yarn. Would you
please advise how to surpass this?{quote}
We can update TestSecureNameNode to include test case for NameNode. This class
already extends SaslDataTransferTestCase.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556270#comment-16556270
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 31s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 30m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 17s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 8s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
28s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 77m 16s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
59s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}328m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestNameNodeMXBean |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12933061/HADOOP-14212.007.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux f16ab6a3634f 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15
17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk /
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555835#comment-16555835
]
Adam Antal commented on HADOOP-14212:
-
Fixed a test related to my modifications (patch v7).
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch,
> HADOOP-14212.007.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554795#comment-16554795
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 32m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 38m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 41s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 30m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 30m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 58s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 42s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
23s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 70m 1s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}346m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
| | hadoop.hdfs.server.datanode.TestDataNodeMXBean |
| | hadoop.hdfs.TestCrcCorruption |
| | hadoop.hdfs.server.namenode.TestFSImageWithXAttr |
| | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.policies.TestDominantResourceFairnessPolicy
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12932899/HADOOP-14212.006.patch
|
| Optional Tests |
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554358#comment-16554358
]
Adam Antal commented on HADOOP-14212:
-
Thank you for the suggestions [~ajayydv], I worked accordingly: removed the
redundant public keywords from the MXBeans, although in TestDataNodeMXBean.java
the contents of testDataNodeMXBean test is in a try-with-resources block from
line 60 to 117, that is why the finally clause got removed - so I see no point
in not removing it.
I added a test for kerberos authentication to TestDataNodeMXBean, could you
please check if my modifications are appropriate? Also another question has
arisen while I attempted to make tests for ResourceManager- and
NodeManagerMXBean. I used SaslDataTransferTestCase as base class for getting
kerberos configuration, but this class is inaccessible from hadoop-yarn. Would
you please advise how to surpass this?
(Moving SaslDataTransferTestCase to hadoop-common is not a good idea since it
uses DFSConfigKeys from hadoop-hdfs, but introducting a new dependency has also
some other caveats.) A simpler answer could be just to copy the code to make
the tests for those two MXBeans in yarn, or don't write test for them.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch, HADOOP-14212.006.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553272#comment-16553272
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 22s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 29s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 16s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
12s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 48s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
53s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}296m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.client.impl.TestBlockReaderLocal |
| | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12932707/HADOOP-14212.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux d75a07909d5a 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552996#comment-16552996
]
Ajay Kumar commented on HADOOP-14212:
-
[~adam.antal] thanks for working on this.
* TestDataNodeMXBean: L114-117 removal of finally block is not required.
* Shall we test case when security is enabled as well? (In current test cases
we are getting default value i.e false)
Checkstyle NITs:
ResourceManagerMXBean,SecondaryNameNodeInfoMXBean, : public modifier is
redundant in function signatures.
DataNodeMxBean: L155 public modifier is redundant.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch, HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552846#comment-16552846
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HADOOP-14212 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12932700/HADOOP-14212.005.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14930/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552833#comment-16552833
]
Adam Antal commented on HADOOP-14212:
-
Reuploaded the same patch to trigger jenkins.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch,
> HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545199#comment-16545199
]
Adam Antal commented on HADOOP-14212:
-
I couldn't recreate the failed test in my computer, so couldn't come up with
new idea about the metrics. Maybe retry jenkins?
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544942#comment-16544942
]
Adam Antal commented on HADOOP-14212:
-
Yes, you're right - I'm looking into it.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543761#comment-16543761
]
Steve Loughran commented on HADOOP-14212:
-
Not sure what is up there; need to see what other jenkins builds are up to.
Some of those stacks look related to metrics, which implies that the changes
may be breaking assertions
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543406#comment-16543406
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 31s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 6s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 11s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 20s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 71m 42s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
51s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}317m 37s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
| | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12931471/HADOOP-14212.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux a25f99edce84 3.13.0-144-generic
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542954#comment-16542954
]
Adam Antal commented on HADOOP-14212:
-
Thanks you Steve, hope its all good.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542891#comment-16542891
]
Steve Loughran commented on HADOOP-14212:
-
LGTM, +1
attached patch 005 for jenkins, which is the one I'll be applying: reorder
imports and add a missing EOF marker nothing significant, but submitting for
due diligence
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch, HADOOP-14212.005.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542095#comment-16542095
]
genericqa commented on HADOOP-14212:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 33m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 53s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 31m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 31m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 9s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 74m
56s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 17m 50s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 64m
10s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}305m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-14212 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12931328/HADOOP-14212.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux cef7a4316183 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / b37074b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs |
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541679#comment-16541679
]
Adam Antal commented on HADOOP-14212:
-
Thank you for looking into it, I used try-with-resources in all the tests
(uploaded patch). Waiting for jenkins now.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch, HADOOP-14212.004.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540241#comment-16540241
]
Steve Loughran commented on HADOOP-14212:
-
I like publishing info as metrics, we use it for S3A perf testing. One concern
of people running things in production are cost of metric creation and
stability. I don't see us removing this.
while jenkins does its testing, quick comment on the tests. in
{{testResourceManagerMXBean}} you should close() the RM afterwards; using a
try-with-resources should do that. thanks
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540214#comment-16540214
]
Adam Antal commented on HADOOP-14212:
-
MXBeans added for ResourceManager and NodeManager, added test too.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch,
> HADOOP-14212.003.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540076#comment-16540076
]
Adam Antal commented on HADOOP-14212:
-
Patch for DN with tests uploaded. Modified test passed in my computer.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch, HADOOP-14212.002.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14212) Expose SecurityEnabled boolean field in JMX for other services besides NameNode
[
https://issues.apache.org/jira/browse/HADOOP-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540003#comment-16540003
]
Adam Antal commented on HADOOP-14212:
-
Added patch for SNN. Unit testing is not applicable (no testcases for
SecondaryNameNodeInfoMXBean)
Continue working on other services.
> Expose SecurityEnabled boolean field in JMX for other services besides
> NameNode
> ---
>
> Key: HADOOP-14212
> URL: https://issues.apache.org/jira/browse/HADOOP-14212
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Burgemeestre
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie, security
> Attachments: HADOOP-14212.001.patch
>
>
> The following commit
> https://github.com/apache/hadoop/commit/dc17bda4b677e30c02c2a9a053895a43e41f7a12
> introduced a "SecurityEnabled" field in the JMX output for the NameNode. I
> believe it would be nice to add this same change to the JMX output of other
> services: Secondary Namenode, ResourceManager, NodeManagers, DataNodes, etc.
> So that it can be queried whether Security is enabled in all JMX resources.
> The reason I am suggesting this feature / improvement is that I think it
> would provide a clean way to check whether your cluster is completely
> Kerberized or not. I don't think there is an easy/clean way to do this now,
> other than checking the logs, checking ports etc.?
> The file where the change was made is
> hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
> has the following function now:
> {code:java}
> @Override // NameNodeStatusMXBean
> public boolean isSecurityEnabled() {
> return UserGroupInformation.isSecurityEnabled();
> }
> {code}
> I would be happy to develop a patch if it seems useful by others as well?
> This is a snippet from the JMX output from the NameNode in case security is
> not enabled:
> {code}
> {
> "name" : "Hadoop:service=NameNode,name=NameNodeStatus",
> "modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
> "NNRole" : "NameNode",
> "HostAndPort" : "node001.cm.cluster:8020",
> "SecurityEnabled" : false,
> "LastHATransitionTime" : 0,
> "State" : "standby"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
