Xiaolin Ha created HBASE-26303:
----------------------------------
Summary: Use priority queue in dir scan pool of cleaner
Key: HBASE-26303
URL: https://issues.apache.org/jira/browse/HBASE-26303
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0, 3.0.0-alpha-1
Reporter: Xiaolin Ha
Assignee: Xiaolin Ha
DirScanPool used normal LinkedBlockingQueue when creating thread pool,
{code:java}
54 private static ThreadPoolExecutor initializePool(int size) {
55 return Threads.getBoundedCachedThreadPool(size, 1, TimeUnit.MINUTES,
56 new
ThreadFactoryBuilder().setNameFormat("dir-scan-pool-%d").setDaemon(true)
57
.setUncaughtExceptionHandler(Threads.LOGGING_EXCEPTION_HANDLER).build());
58 }
{code}
which will not priority scan larger directories and delete files there as
expected, though CleanerChore#sortByConsumedSpace() before putting directories
to the queue.
Subdirectories of larger directories and small directories are taken fairly in
the queue.
We should used priority queue here instead, e.g. PriorityBlockingQueue, to make
larger directories be cleaned earlier.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
