[
https://issues.apache.org/jira/browse/HDFS-16528?focusedWorklogId=760535&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-760535
]
ASF GitHub Bot logged work on HDFS-16528:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Apr/22 01:16
Start Date: 22/Apr/22 01:16
Worklog Time Spent: 10m
Work Description: tomscut commented on code in PR #4186:
URL: https://github.com/apache/hadoop/pull/4186#discussion_r855696796
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java:
##########
@@ -1856,15 +1875,16 @@ public DatanodeCommand[]
handleHeartbeat(DatanodeRegistration nodeReg,
nodeinfo.setBalancerBandwidth(0);
}
- if (slowPeerTracker != null) {
- final Map<String, Double> slowPeersMap = slowPeers.getSlowPeers();
- if (!slowPeersMap.isEmpty()) {
- if (LOG.isDebugEnabled()) {
- LOG.debug("DataNode " + nodeReg + " reported slow peers: " +
- slowPeersMap);
- }
- for (String slowNodeId : slowPeersMap.keySet()) {
- slowPeerTracker.addReport(slowNodeId, nodeReg.getIpcAddr(false));
+ synchronized (slowPeerTrackerLock) {
Review Comment:
Thanks @virajjasani for updating. Adding lock `slowPeerTrackerLock` looks
good. But I'm not sure if it affects heartbeat here.
I have one small suggestion:
1. Make dataNodePeerStatsEnabled volatile.
2. When dataNodePeerStatsEnabledVal = false, do not set slowPeerTracker to
null.
3. Change the condition from `if (slowPeerTracker != null)` to `if
(slowPeerTracker != null) && dataNodePeerStatsEnabled` where `slowPeerTracker`
is used.
Disadvantages:
if `dataNodePeerStatsEnabledVal = false`, `slowPeerTracker` will be
retained. But I guess the operation to `disable slowPeerTracker` is not common.
Advantages:
Does not cause lock contention and does not cause NPE.
What do you think of this?
Issue Time Tracking
-------------------
Worklog Id: (was: 760535)
Time Spent: 3h (was: 2h 50m)
> Reconfigure slow peer enable for Namenode
> -----------------------------------------
>
> Key: HDFS-16528
> URL: https://issues.apache.org/jira/browse/HDFS-16528
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> HDFS-16396 provides reconfig options for several configs associated with
> slownodes in Datanode. Similarly, HDFS-16287 and HDFS-16327 have added some
> slownodes related configs as the reconfig options in Namenode.
> The purpose of this Jira is to add DFS_DATANODE_PEER_STATS_ENABLED_KEY as
> reconfigurable option for Namenode (similar to how HDFS-16396 has included it
> for Datanode).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
