[
https://issues.apache.org/jira/browse/HDFS-15039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041583#comment-17041583
]
Lisheng Sun edited comment on HDFS-15039 at 2/21/20 6:49 AM:
-------------------------------------------------------------
Such the problem that excessive io operations does exist
But there are also io operations in LocalReplica#getMetaFile of this patch.
HDFS-15174 optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io
operations. e.g. Get block size by ReplicaInfo and meta size by
DataChecksum#getChecksumSize().
was (Author: leosun08):
LGTM.
Such the problem that excessive io operations does exist
HDFS-15174 optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io
operations. e.g. Get block size by ReplicaInfo and meta size by
DataChecksum#getChecksumSize().
> Cache meta file length of FinalizedReplica to reduce call File.length()
> -----------------------------------------------------------------------
>
> Key: HDFS-15039
> URL: https://issues.apache.org/jira/browse/HDFS-15039
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15039.patch
>
>
> When use ReplicaCachingGetSpaceUsed to get the volume space used. It will
> call File.length() for every meta file of replica. That add more disk IO, we
> found the slow log as below. For finalized replica, the size of meta file is
> not changed, i think we can cache the value.
> {code:java}
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
> Refresh dfs used, bpid: BP-898717543-10.75.1.240-1519386995727 replicas
> size: 1166 dfsUsed: 72227113183 on volume:
> DS-3add8d62-d69a-4f5a-a29f-b7bbb400af2e duration: 17206ms{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
