Yu Li created HBASE-17215:
-----------------------------
Summary: Separate small/large file delete threads in HFileCleaner
to accelerate hfile cleanup speed
Key: HBASE-17215
URL: https://issues.apache.org/jira/browse/HBASE-17215
Project: HBase
Issue Type: Improvement
Reporter: Yu Li
Assignee: Yu Li
When using PCIe-SSD the flush speed will be really quick, and although we have
per CF flush, we still have the
{{hbase.regionserver.optionalcacheflushinterval}} setting and some other
mechanism to avoid data kept in memory for too long to flush small hfiles. In
our online environment we found the single thread cleaner kept cleaning earlier
flushed small files while large files got no chance, which caused disk full
then many other problems.
Deleting hfiles in parallel with too many threads will also increase the
workload of namenode, so here we propose to separate large/small hfile cleaner
threads just like we do for compaction, and it turned out to work well in our
cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
