Nathan Roberts created HDFS-8873:
------------------------------------
Summary: throttle directoryScanner
Key: HDFS-8873
URL: https://issues.apache.org/jira/browse/HDFS-8873
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.7.1
Reporter: Nathan Roberts
The new 2-level directory layout can make directory scans expensive in terms of
disk seeks (see HDFS-8791) for details.
It would be good if the directoryScanner() had a configurable duty cycle that
would reduce its impact on disk performance (much like the approach in
HDFS-8617).
Without such a throttle, disks can go 100% busy for many minutes at a time
(assuming the common case of all inodes in cache but no directory blocks
cached, 64K seeks are required for full directory listing which translates to
655 seconds)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
