[
https://issues.apache.org/jira/browse/HDFS-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18097302#comment-18097302
]
ASF GitHub Bot commented on HDFS-17890:
---------------------------------------
junjie1233 opened a new pull request, #8616:
URL: https://github.com/apache/hadoop/pull/8616
HDFS-17890: Avoid slow disks datanode when reading data.
(Supersedes #8338, which was closed to reset the CI history that had
accumulated flaky-test failures across trunk drift / runner load. The change is
identical.)
## Change
Deprioritize replicas on slow disks when returning block locations to
clients, so reads prefer faster storage. Slow disks are detected via existing
DataNode heartbeat reports and cached on the NameNode for efficient read-path
lookup.
- `dfs.namenode.deprioritize.slow.disk.datanode.for.read` (default `false`)
gates the read-path reordering; the slow-disk cache rebuild only runs when the
feature is enabled (zero overhead when disabled).
- `dfs.namenode.slow.disk.cache.rebuild.interval` (default `30s`) controls
cache rebuild cadence.
- Stable sort preserves network-topology order for non-slow replicas;
erasure-coded blocks are skipped.
- Disk tracking is keyed on StorageID (`volumePath|storageID` format) for
per-disk granularity.
## Tests
`TestSlowDiskBlockLocations` (7 tests) covers cache rebuild, expiration, and
read-path sorting.
> Avoid slow disks datanode when reading data
> -------------------------------------------
>
> Key: HDFS-17890
> URL: https://issues.apache.org/jira/browse/HDFS-17890
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
> Reporter: JunJieZhao
> Priority: Major
> Labels: pull-request-available
>
> During HDFS read operations, replicas on slow disks should be deprioritized
> so that replicas on healthy disks are preferred.
> Currently, slow disk avoidance is not performed at the disk level when
> reading operations return to the block position.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
