[
https://issues.apache.org/jira/browse/HDFS-17360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812792#comment-17812792
]
ASF GitHub Bot commented on HDFS-17360:
---------------------------------------
huangzhaobo99 commented on PR #6505:
URL: https://github.com/apache/hadoop/pull/6505#issuecomment-1919480906
> @huangzhaobo99 I have a question. If the IO of the machine where the DN is
located is abnormal, causing exceptions in many blocks, what will this metric
look like?
>
>
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6505/7/artifact/out/branch-mvninstall-root.txt
>
> We should fix mvn install, we need to re-trigger compilation.
Hi @slfan1989, Is the IO exception mentioned here a fault with DN?
If so, There are currently relevant exception handling mechanisms in place,
there is to ensure that the elements stored in the map set are reasonable,
including the following points:
1. The readBlock method adds 1 to the blockId before reading data, and
subtracts 1 from the blockId when it executes normally or throws an exception.
2. The maximum number of read threads on a DN is close to the configuration
of the xciver thread. When there is an exception in the read block, the total
value in the map will not exceed the number of resident xciver threads.
3. When there is no read request, this map is an empty set of maps.
In addition, the ReadBlockIdCounts metric and the xciver thread metric are
used together, when a sudden increase in xciver threads is detected and lasts
for 2 or 3 minutes, the map can be used to locate the block that has been
abnormally accessed.
> Record the number of times a block is read during a certain time period.
> ------------------------------------------------------------------------
>
> Key: HDFS-17360
> URL: https://issues.apache.org/jira/browse/HDFS-17360
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: huangzhaobo
> Assignee: huangzhaobo
> Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
