[
https://issues.apache.org/jira/browse/HDFS-16648?focusedWorklogId=788140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-788140
]
ASF GitHub Bot logged work on HDFS-16648:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Jul/22 05:26
Start Date: 06/Jul/22 05:26
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4529:
URL: https://github.com/apache/hadoop/pull/4529#issuecomment-1175795921
> What are we saving by doing this?
In [PR-4480](https://github.com/apache/hadoop/pull/4480#event-6923776715), I
do a performance stress test about it. And found that adding isDebugEnabled
before strings concatenation is helpful for performance.
Please refer to the stress test code and results.
Test code:
```
org.slf4j.Logger LOG = LoggerFactory.getLogger("Test");
Block b = new Block();
DatanodeInfo dn = new DatanodeInfo(EMPTY_DATANODE_ID);
List<DatanodeInfo> staleNodes = new ArrayList<>();
staleNodes.add(dn);
long beginTime1 = Time.monotonicNowNanos();
for (int i = 0; i < 100000000; i++) {
LOG.debug("BLOCK* invalidateBlocks: postponing " +
"invalidation of {} on {} because {} replica(s) are located on
" +
"nodes with potentially out-of-date block reports",
b, dn, staleNodes);
}
long endTime1 = Time.monotonicNowNanos();
LOG.info("Time1: {}", endTime1 - beginTime1);
long beginTime2 = Time.monotonicNowNanos();
for (int i = 0; i < 100000000; i++) {
if (LOG.isDebugEnabled()) {
LOG.debug("BLOCK* invalidateBlocks: postponing " +
"invalidation of {} on {} because {} replica(s) are located
on " +
"nodes with potentially out-of-date block reports",
b, dn, staleNodes);
}
}
long endTime2 = Time.monotonicNowNanos();
LOG.info("Time2: {}", endTime2 - beginTime2);
long beginTime3 = Time.monotonicNowNanos();
for (int i = 0; i < 100000000; i++) {
LOG.debug("BLOCK* invalidateBlocks: postponing invalidation of {}", b);
}
long endTime3 = Time.monotonicNowNanos();
LOG.info("Time3: {}", endTime3 - beginTime3);
```
And the result is:
| - | Time1(ns) | Time2(ns) | Time3(ns) |
| :-----| ----: | :----: | :----: |
| First Run | 353681417 | 287066916 | 270287253 |
| Second Run | 356797081 | 308013207 | 266873810 |
| Third Run | 366995010 | 287111940 | 259907404 |
| Avg | 359157836 | 294064021 | 265689489 |
Issue Time Tracking
-------------------
Worklog Id: (was: 788140)
Time Spent: 40m (was: 0.5h)
> Normalize the usage of debug logs in NameNode
> ---------------------------------------------
>
> Key: HDFS-16648
> URL: https://issues.apache.org/jira/browse/HDFS-16648
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> There are many irregular debug logs in NameNode. such as:
> Error type1:
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.debug("Getting groups for user " + user);
> }
> {code}
> we can format it to:
> {code:java}
> LOG.debug("Getting groups for user {}. ", user);
> {code}
> Error type2:
> {code:java}
> LOG.debug("*DIR* NameNode.renameSnapshot: Snapshot Path {}, " +
> "snapshotOldName {}, snapshotNewName {}", snapshotRoot,
> snapshotOldName, snapshotNewName);
> {code}
> we can format it to:
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.debug("*DIR* NameNode.renameSnapshot: Snapshot Path {}, " +
> "snapshotOldName {}, snapshotNewName {}", snapshotRoot,
> snapshotOldName, snapshotNewName);
> }
> {code}
> Error type3:
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.debug("getAdditionalDatanode: src=" + src
> + ", fileId=" + fileId
> + ", blk=" + blk
> + ", existings=" + Arrays.asList(existings)
> + ", excludes=" + Arrays.asList(excludes)
> + ", numAdditionalNodes=" + numAdditionalNodes
> + ", clientName=" + clientName);
> }
> {code}
> We can format it to:
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.debug("getAdditionalDatanode: src={}, fileId={}, "
> + "blk={}, existings={}, excludes={}, numAdditionalNodes={}, "
> + "clientName={}.", src, fileId, blk, Arrays.asList(existings),
> Arrays.asList(excludes), numAdditionalNodes, clientName);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
