[
https://issues.apache.org/jira/browse/HDFS-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17124717#comment-17124717
]
Yang Yun edited comment on HDFS-15377 at 6/3/20, 7:46 AM:
----------------------------------------------------------
Thanks [~liuml07] for the question.
the dfs.block.scanner.part.size is suggested to be the multiple of
dfs.bytes-per-checksum.
was (Author: hadoop_yangyun):
Thanks [~liuml07] for the question.
> BlockScanner scans one part per round, expect full scans after several rounds
> -----------------------------------------------------------------------------
>
> Key: HDFS-15377
> URL: https://issues.apache.org/jira/browse/HDFS-15377
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15377.002.patch, HDFS-15377.003.patch,
> HDFS-15377.004.patch, HDFS-15377.004.patch
>
>
> For reducing disk IO, one block is separated to multiple parts, BlockScanner
> scans only one part per round. Expect that after several rounds, the full
> block should be scanned
> Add a new option "dfs.block.scanner.part.size". the maximum data size per
> scan by the block scanner. this value should be the multiple of chunk size,
> for example, 512, 1024, 4096 ...
> Default value is -1, will disable partial scan.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
