[
https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708868#comment-14708868
]
Rui Li commented on HDFS-8920:
------------------------------
Hi [~cmccabe] and [~zhz], thanks for your inputs. I agree that it makes sense
to warn the user when blocks get lost. Problem is that the loss is more likely
to happen in EC mode which makes the warning more frequent. Besides Zhe's
suggestion, another possible solution is to only print the warning when we try
to get a striped block from a dead DN for 1st time. To do so, maybe we need
some data structure to remember which nodes have been warned.
Which do you prefer? Or any other suggestions?
> Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt
> performance
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-8920
> URL: https://issues.apache.org/jira/browse/HDFS-8920
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Rui Li
> Assignee: Rui Li
>
> When we test reading data with datanodes killed,
> {{DFSInputStream::getBestNodeDNAddrPair}} becomes a hot spot method and
> effectively blocks the client JVM. This log seems too verbose:
> {code}
> if (chosenNode == null) {
> DFSClient.LOG.warn("No live nodes contain block " + block.getBlock() +
> " after checking nodes = " + Arrays.toString(nodes) +
> ", ignoredNodes = " + ignoredNodes);
> return null;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
