[
https://issues.apache.org/jira/browse/HDFS-16521?focusedWorklogId=750012&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750012
]
ASF GitHub Bot logged work on HDFS-16521:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Mar/22 11:08
Start Date: 30/Mar/22 11:08
Worklog Time Spent: 10m
Work Description: iwasakims commented on pull request #4107:
URL: https://github.com/apache/hadoop/pull/4107#issuecomment-1082994545
I agree with @ayushtkn that modifying ClientProtocol is overkill for the use
case. @virajjasani
> While I agree that JMX metric for slownode is already available, not every
downstreamer might have access to it directly, for instance in K8S managed
clusters, unless port forward is enabled (not so common case in prod), HDFS
downstreamer would not be able to access JMX metrics.
Thanks to
[JMXJsonServlet](https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/jmx/JMXJsonServlet.java),
we can get metrics in JSON format via HTTP/HTTPS port of NameNode without
additional configuration. JSON on HTTP is usually easier to access from
outside/downstream than Protobuf on RPC.
```
$ curl namenode:9870/jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus
{
"beans" : [ {
"name" : "Hadoop:service=NameNode,name=NameNodeStatus",
"modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
"NNRole" : "NameNode",
"HostAndPort" : "localhost:8020",
"SecurityEnabled" : false,
"LastHATransitionTime" : 0,
"BytesWithFutureGenerationStamps" : 0,
"SlowPeersReport" : "[]",
"SlowDisksReport" : null,
"State" : "active"
} ]
}
```
How about enhancing metrics if the current information in the
SlowPeersReport is insufficient?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 750012)
Time Spent: 1h 10m (was: 1h)
> DFS API to retrieve slow datanodes
> ----------------------------------
>
> Key: HDFS-16521
> URL: https://issues.apache.org/jira/browse/HDFS-16521
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In order to build some automation around slow datanodes that regularly show
> up in the slow peer tracking report, e.g. decommission such nodes and queue
> them up for external processing and add them back later to the cluster after
> fixing issues etc, we should expose DFS API to retrieve all slow nodes at a
> given time.
> Providing such API would also help add an additional option to "dfsadmin
> -report" that lists slow datanodes info for operators to take a look,
> specifically useful filter for larger clusters.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
