[
https://issues.apache.org/jira/browse/HBASE-30384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-30384 started by Vladimir Rodionov.
-------------------------------------------------
> Add dynamic cache sizing support for CacheEngine-based caches
> -------------------------------------------------------------
>
> Key: HBASE-30384
> URL: https://issues.apache.org/jira/browse/HBASE-30384
> Project: HBase
> Issue Type: New Feature
> Components: BlockCache
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Priority: Major
> Fix For: 4.0.0-alpha-1
>
>
> h2. Description
> The new block cache architecture represents cache implementations through
> {{CacheEngine}}, but
> {{CacheEngine}} currently has no capability for changing the maximum cache
> size at runtime.
> Dynamic cache sizing is required by HBase heap-memory tuning. In the legacy
> cache architecture this
> capability is represented by {{ResizableBlockCache}} and {{setMaxSize(long)}}.
> {{CombinedBlockCache}} also implements {{ResizableBlockCache}} and applies
> dynamic resizing to its
> first-level cache only. This preserves the expected heap-memory tuning
> behavior for tiered legacy
> caches.
> The new cache architecture needs an equivalent capability so that native
> {{CacheEngine}}
> implementations can participate in dynamic cache sizing after RegionServer
> cache ownership is
> migrated to {{CacheAccessService}}.
> Dynamic resizing should be represented as an optional cache engine capability
> rather than adding
> {{setMaxSize()}} to every {{CacheEngine}} implementation.
> Introduce a capability interface for cache engines that support runtime
> resizing, for example:
> {code:java}
> @InterfaceAudience.Private
> public interface ResizableCacheEngine extends CacheEngine {
> /**
> * Changes the maximum size of this cache engine.
> *
> * @param maxSize new maximum cache size in bytes
> */
> void setMaxSize(long maxSize);
> }
> {code}
> Native cache engines that support dynamic resizing, such as
> {{LruCacheEngine}}, should implement
> this interface.
> This issue establishes the cache-engine resizing capability only. Migration of
> {{HeapMemoryManager}} to the new cache architecture requires migration of
> {{HRegionServer}} cache
> ownership from {{BlockCache}} to {{CacheAccessService}} and is therefore
> outside the scope of this
> issue.
> h2. Current Architecture
> {{HeapMemoryManager}} is currently created by {{HRegionServer}} using the
> legacy {{BlockCache}}
> instance.
> Dynamic cache sizing therefore follows the legacy cache hierarchy:
> {noformat}
> HRegionServer
> |
> v
> BlockCache
> |
> v
> HeapMemoryManager
> |
> v
> ResizableBlockCache
> |
> v
> setMaxSize(...)
> {noformat}
> For {{CombinedBlockCache}}, the resizing operation applies to the first-level
> cache while the
> second-level cache remains unchanged.
> This behavior should remain unchanged until {{HRegionServer}} and
> {{HeapMemoryManager}} are migrated
> to the new cache architecture.
> h2. Target Capability
> The new architecture should provide an equivalent optional resizing
> capability:
> {noformat}
> CacheEngine
> |
> v
> ResizableCacheEngine
> |
> v
> setMaxSize(...)
> {noformat}
> A cache engine that supports runtime resizing implements
> {{ResizableCacheEngine}}.
> A cache engine that does not support dynamic resizing does not implement the
> interface and does not
> need to provide a no-op {{setMaxSize()}} implementation.
> h2. Proposed Changes
> * Introduce {{ResizableCacheEngine}} as an optional {{CacheEngine}}
> capability.
> * Make {{LruCacheEngine}} implement {{ResizableCacheEngine}}.
> * Preserve the existing {{LruCacheEngine.setMaxSize(long)}} behavior.
> * Preserve dynamic-resizing capability when legacy {{ResizableBlockCache}}
> implementations are
> adapted to {{CacheEngine}}, where applicable.
> * Add tests for resizing a native {{LruCacheEngine}} through
> {{ResizableCacheEngine}}.
> * Add tests for preservation of resizing capability through legacy cache
> adapters, where
> applicable.
> h2. Future Integration
> After {{HRegionServer}} cache ownership is migrated from {{BlockCache}} to
> {{CacheAccessService}}, {{HeapMemoryManager}} can be migrated to operate on
> {{ResizableCacheEngine}} instead of {{ResizableBlockCache}}.
> At that point, selection of the cache engine participating in heap-memory
> tuning should preserve the
> existing semantics:
> * For a single-tier topology, resize the {{SINGLE}} engine when it implements
> {{ResizableCacheEngine}}.
> * For a tiered topology, resize the heap-resident {{L1}} engine when it
> implements
> {{ResizableCacheEngine}}.
> * {{L2}} should not be resized merely because it belongs to the same topology.
> That integration is intentionally deferred to the
> RegionServer/{{CacheAccessService}} ownership
> migration.
> h2. Out of Scope
> * Changes to {{HRegionServer}} cache ownership.
> * Migration of {{HeapMemoryManager}} from
> {{BlockCache}}/{{ResizableBlockCache}} to
> {{CacheAccessService}}/{{ResizableCacheEngine}}.
> * Changes to {{CombinedBlockCache}} resizing behavior.
> * Changes to the {{HeapMemoryTuner}} algorithm or tuning policy.
> * General configuration-change propagation through {{CacheAccessService}}.
> * Migration of {{BucketCache}} to a native {{CacheEngine}} implementation.
> * Changes to cache admission, placement, promotion, or eviction policy.
> * Making every {{CacheEngine}} dynamically resizable.
> * Topology-level selection of the engine that participates in heap-memory
> tuning.
> h2. Acceptance Criteria
> * {{ResizableCacheEngine}} represents the optional runtime-resizing
> capability for
> {{CacheEngine}} implementations.
> * {{LruCacheEngine}} implements {{ResizableCacheEngine}}.
> * A native {{LruCacheEngine}} can be resized at runtime through
> {{ResizableCacheEngine.setMaxSize(long)}}.
> * Existing {{LruCacheEngine.setMaxSize(long)}} behavior is preserved.
> * Cache engines that do not support dynamic resizing are not required to
> implement
> {{setMaxSize()}}.
> * Legacy {{ResizableBlockCache}} implementations retain their resizing
> capability when adapted to
> {{CacheEngine}}, where applicable.
> * Existing {{HeapMemoryManager}}, {{CombinedBlockCache}}, and legacy
> heap-memory tuning behavior
> remain unchanged.
> * Relevant cache-engine and block-cache tests continue to pass.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
