[
https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17726709#comment-17726709
]
ASF GitHub Bot commented on HDFS-17026:
---------------------------------------
hchaverri commented on code in PR #5691:
URL: https://github.com/apache/hadoop/pull/5691#discussion_r1207175135
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java:
##########
@@ -107,6 +110,15 @@ public class NamenodeHeartbeatService extends
PeriodicService {
/** URL scheme to use for JMX calls. */
private String scheme;
+ /** Frequency of updates to JMX report. */
+ private long updateJmxIntervalMs;
+ /** Timestamp of last attempt to update JMX report. */
+ private long lastJmxUpdateAttempt;
Review Comment:
These metrics are being used but not in a critical path for all cases (see
[RBFMetrics](https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java)
or
[AvailableSpaceResolver](https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/order/AvailableSpaceResolver.java)
for examples). This PR only gives the option to change the frequency to reduce
impact on the Namenode
> RBF: NamenodeHeartbeatService should update JMX report with configurable
> frequency
> ----------------------------------------------------------------------------------
>
> Key: HDFS-17026
> URL: https://issues.apache.org/jira/browse/HDFS-17026
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
> Reporter: Hector Sandoval Chaverri
> Priority: Major
> Labels: pull-request-available
>
> The NamenodeHeartbeatService currently calls each of the Namenode's JMX
> endpoint every time it wakes up (default value is every 5 seconds).
> In a cluster with 40 routers, we have observed service degradation on some of
> the Namenodes, since the JMX request obtains Datanode status and blocks
> other RPC requests. However, JMX report data doesn't seem to be used for
> critical paths on the routers.
> We should configure the NamenodeHeartbeatService so it updates the JMX
> reports on a slower frequency than the Namenode states or to disable the
> reports completely.
> The class calls out the JMX request being optional even though there is no
> implementation to turn it off:
> {noformat}
> // Read the stats from JMX (optional)
> updateJMXParameters(webAddress, report);{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
