Andrew Kyle Purtell created HBASE-24428:
-------------------------------------------
Summary: Priority compaction for recently split daughter regions
Key: HBASE-24428
URL: https://issues.apache.org/jira/browse/HBASE-24428
Project: HBase
Issue Type: Improvement
Components: Compaction
Reporter: Andrew Kyle Purtell
We observe that under hotspotting conditions that splitting will proceed very
slowly and the "_Cannot split region due to reference files being there_" log
line will be logged excessively. (branch-1 based production.) This is because
after a region is split it must be compacted before it can be split again.
Reference files must be replaced by real HFiles, normal housekeeping performed
during compaction. However if the regionserver is under excessive load, its
compaction queues may become deep. The daughters of a recently split
hotspotting region may themselves continue to hotspot and will rapidly need to
split again. If the scheduled compaction work to remove/replace reference files
is queued hundreds or thousands of compaction queue elements behind current,
the recently split daughter regions will not be able to split again for a long
time and may grow very large, producing additional complications (very large
regions, very deep replication queues).
To help avoid this condition we should prioritize the compaction of recently
split daughter regions. Compaction requests include a {{priority}} field and
CompactionRequest implements a comparator that sorts by this field. We already
detect when a compaction request involves a region that has reference files, to
ensure that it gets selected to be eligible for compaction, but we do not seem
to prioritize the requests for post-split housekeeping. Split work should be
placed at the top of the queue. Ensure that this is happening.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
