[
https://issues.apache.org/jira/browse/HDFS-16316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468540#comment-17468540
]
JiangHua Zhu edited comment on HDFS-16316 at 1/4/22, 11:18 AM:
---------------------------------------------------------------
This problem is related to the persistent size of the dfsused file.
In general, there are two types of data that will be persisted to dfsused:
1. The size of all blk_xxx and blk_xxxx.meta files under
FsVolume->BlockPoolSlice.
2. Time.
When the file size is wrong, this will cause problems for all subsequent
displays. For example, the size of the dfsused file included here is:
694649413944697
Converted to TB = 631.78. This is obviously incorrect.
!screenshot-4.png!
We can try to fix:
When BlockPool is initialized, the value of CachingGetSpaceUsed#used is reset.
was (Author: jianghuazhu):
This problem is related to the size of the persistence to the dfsused file.
Under normal circumstances, there are two types of data will be persisted to
dfsused:
1. The size of all blk_xxx and blk_xxxx.meta files under
FsVolume->BlockPoolSlice.
2. Time.
When the file size is wrong, this will cause problems for all subsequent
displays. For example, the size of the dfsused file included here is:
694649413944697
Converted to TB = 631.78. This is obviously incorrect.
!screenshot-4.png!
We can try to fix:
1. When the DN is started, when the ReplicationMap is initialized, the value of
CachingGetSpaceUsed#used is reset.
> Improve DataNode to calculate used capacity
> -------------------------------------------
>
> Key: HDFS-16316
> URL: https://issues.apache.org/jira/browse/HDFS-16316
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.9.2
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png,
> screenshot-4.png
>
>
> Something unusual happened in the online environment.
> The DataNode is configured with 11 disks (${dfs.datanode.data.dir}). It is
> normal for 10 disks to calculate the used capacity, and the calculated value
> for the other 1 disk is much larger, which is very strange.
> This is about the live view on the NameNode:
> !screenshot-1.png!
> This is about the live view on the DataNode:
> !screenshot-2.png!
> We can look at the view on linux:
> !screenshot-3.png!
> There is a big gap here, regarding'/mnt/dfs/11/data'. This situation should
> be prohibited from happening.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
