[jira] [Updated] (HDFS-15174) Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations
[ https://issues.apache.org/jira/browse/HDFS-15174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-15174: --- Resolution: Fixed Status: Resolved (was: Patch Available) Patch applies cleanly in trunk branch-3.2 and branch-3.1. Branch-2.10 will require an update. > Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations > - > > Key: HDFS-15174 > URL: https://issues.apache.org/jira/browse/HDFS-15174 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Lisheng Sun >Assignee: Lisheng Sun >Priority: Major > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: HDFS-15174-001.patch > > > Calculating the size of each block and the size of the meta file requires io > operation In ReplicaCachingGetSpaceUsed#refresh(). Pressure on disk > performance when there are many block. HDFS-14313 is intended to reduce io > operation. So get block size by ReplicaInfo and meta size by > DataChecksum#getChecksumSize(). > {code:java} > @Override > protected void refresh() { > if (CollectionUtils.isNotEmpty(replicaInfos)) { > for (ReplicaInfo replicaInfo : replicaInfos) { > if (Objects.equals(replicaInfo.getVolume().getStorageID(), > volume.getStorageID())) { > dfsUsed += replicaInfo.getBlockDataLength(); > dfsUsed += replicaInfo.getMetadataLength(); > count++; > } > } > } > } > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15174) Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations
[ https://issues.apache.org/jira/browse/HDFS-15174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-15174: --- Fix Version/s: 3.2.2 3.1.4 3.3.0 > Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations > - > > Key: HDFS-15174 > URL: https://issues.apache.org/jira/browse/HDFS-15174 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Lisheng Sun >Assignee: Lisheng Sun >Priority: Major > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: HDFS-15174-001.patch > > > Calculating the size of each block and the size of the meta file requires io > operation In ReplicaCachingGetSpaceUsed#refresh(). Pressure on disk > performance when there are many block. HDFS-14313 is intended to reduce io > operation. So get block size by ReplicaInfo and meta size by > DataChecksum#getChecksumSize(). > {code:java} > @Override > protected void refresh() { > if (CollectionUtils.isNotEmpty(replicaInfos)) { > for (ReplicaInfo replicaInfo : replicaInfos) { > if (Objects.equals(replicaInfo.getVolume().getStorageID(), > volume.getStorageID())) { > dfsUsed += replicaInfo.getBlockDataLength(); > dfsUsed += replicaInfo.getMetadataLength(); > count++; > } > } > } > } > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15174) Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations
[ https://issues.apache.org/jira/browse/HDFS-15174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15174: --- Attachment: HDFS-15174-001.patch Status: Patch Available (was: Open) > Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations > - > > Key: HDFS-15174 > URL: https://issues.apache.org/jira/browse/HDFS-15174 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Lisheng Sun >Assignee: Lisheng Sun >Priority: Major > Attachments: HDFS-15174-001.patch > > > Calculating the size of each block and the size of the meta file requires io > operation In ReplicaCachingGetSpaceUsed#refresh(). Pressure on disk > performance when there are many block. HDFS-14313 is intended to reduce io > operation. So get block size by ReplicaInfo and meta size by > DataChecksum#getChecksumSize(). > {code:java} > @Override > protected void refresh() { > if (CollectionUtils.isNotEmpty(replicaInfos)) { > for (ReplicaInfo replicaInfo : replicaInfos) { > if (Objects.equals(replicaInfo.getVolume().getStorageID(), > volume.getStorageID())) { > dfsUsed += replicaInfo.getBlockDataLength(); > dfsUsed += replicaInfo.getMetadataLength(); > count++; > } > } > } > } > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org