Jacob LeBlanc created HBASE-23066:
-------------------------------------
Summary: Allow cache on write during compactions when prefetching
is enabled
Key: HBASE-23066
URL: https://issues.apache.org/jira/browse/HBASE-23066
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 1.4.10
Reporter: Jacob LeBlanc
Fix For: 1.4.11
Attachments: prefetchCompactedBlocksOnWrite.patch
In cases where users care a lot about read performance for tables that are
small enough to fit into a cache (or the cache is large enough), prefetchOnOpen
can be enabled to make the entire table available in cache after the initial
region opening is completed. Any new data can also be guaranteed to be in cache
with the cacheBlocksOnWrite setting.
However, the missing piece is when all blocks are evicted after a compaction.
We found very poor performance after compactions for tables under heavy read
load and a slower backing filesystem (S3). After a compaction the prefetching
threads need to compete with threads servicing read requests and get constantly
blocked as a result.
This is a proposal to introduce a new cache configuration option that would
cache blocks on write during compaction for any column family that has prefetch
enabled. This would virtually guarantee all blocks are kept in cache after the
initial prefetch on open is completed allowing for guaranteed steady read
performance despite a slow backing file system.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
