[
https://issues.apache.org/jira/browse/HDFS-15039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325577#comment-17325577
]
wangyi commented on HDFS-15039:
-------------------------------
If datanode is restart , meta file length will load from disk again ? I
understand right ？Why not load meta file length during datanode startup and
record block meta file length in volumeMap when block is finalized?
> Cache meta file length of FinalizedReplica to reduce call File.length()
> -----------------------------------------------------------------------
>
> Key: HDFS-15039
> URL: https://issues.apache.org/jira/browse/HDFS-15039
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HDFS-15039.006.patch, HDFS-15039.patch,
> HDFS-15039.patch, HDFS-15039.patch, HDFS-15039.patch, HDFS-15039.patch
>
>
> When use ReplicaCachingGetSpaceUsed to get the volume space used. It will
> call File.length() for every meta file of replica. That add more disk IO, we
> found the slow log as below. For finalized replica, the size of meta file is
> not changed, i think we can cache the value.
> {code:java}
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
> Refresh dfs used, bpid: BP-898717543-10.75.1.240-1519386995727 replicas
> size: 1166 dfsUsed: 72227113183 on volume:
> DS-3add8d62-d69a-4f5a-a29f-b7bbb400af2e duration: 17206ms{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
