[
https://issues.apache.org/jira/browse/HDFS-10941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657612#comment-15657612
]
Xiaoyu Yao commented on HDFS-10941:
-----------------------------------
Thanks [~vagarychen] for the update. The v3 patch has a potential perf issue
with the wrapper approach. The toString() and string concat cost will always be
there even with the {{if (LOG.isTraceEnabled())}} guard inside the wrapper.
I would suggest we leverage the slf4j parameterized logging like below to avoid
it without the wrapper.
More detail about sl4fj logging performance can be found here:
http://www.slf4j.org/faq.html#logging_performance.
{code}
case UNDER_REPLICATED:
LOG.trace("under replicated block: {} result: {}", block, res);
nrUnderReplicated++;
{code}
> Improve BlockManager#processMisReplicatesAsync log
> --------------------------------------------------
>
> Key: HDFS-10941
> URL: https://issues.apache.org/jira/browse/HDFS-10941
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Xiaoyu Yao
> Assignee: Chen Liang
> Attachments: HDFS-10941.001.patch, HDFS-10941.002.patch,
> HDFS-10941.002.patch
>
>
> BlockManager#processMisReplicatesAsync is the daemon thread running inside
> namenode to handle miserplicated blocks. As shown below, it has a trace log
> for each of the block in the cluster being processed (10000 blocks per
> iteration after sleep 10s).
> {code}
> MisReplicationResult res = processMisReplicatedBlock(block);
> if (LOG.isTraceEnabled()) {
> LOG.trace("block " + block + ": " + res);
> }
> {code}
> However, it is not very useful as dumping every block in the cluster will
> overwhelm the namenode log without much useful information assuming the
> majority of the blocks are not over/under replicated. This ticket is opened
> to improve the log for easy troubleshooting of block replication related
> issues by:
>
> 1) add debug log for blocks that get under/over replicated result during
> {{processMisReplicatedBlock()}}
> 2) or change to trace log for only blocks that get non-OK result during
> {{processMisReplicatedBlock()}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
