voonhous commented on code in PR #13724:
URL: https://github.com/apache/hudi/pull/13724#discussion_r2335537490
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -527,6 +527,33 @@ public final class HoodieMetadataConfig extends
HoodieConfig {
.withDocumentation("Max size in MB below which metadata file (HFile)
will be downloaded "
+ "and cached entirely for the HFileReader.");
+ public static final ConfigProperty<Boolean>
METADATA_HFILE_BLOCK_CACHE_ENABLED = ConfigProperty
+ .key(METADATA_PREFIX + ".hfile.block.cache.enabled")
+ .defaultValue(false)
+ .markAdvanced()
+ .sinceVersion("1.1.0")
+ .withDocumentation("Enable HFile block-level caching for metadata files.
This caches frequently "
+ + "accessed HFile blocks in memory to reduce I/O operations during
metadata queries. "
+ + "Improves performance for workloads with repeated metadata access
patterns.");
+
+ public static final ConfigProperty<Integer> METADATA_HFILE_BLOCK_CACHE_SIZE
= ConfigProperty
+ .key(METADATA_PREFIX + ".hfile.block.cache.size")
+ .defaultValue(100)
+ .markAdvanced()
+ .sinceVersion("1.1.0")
+ .withDocumentation("Maximum number of HFile blocks to cache in memory
per metadata file reader. "
+ + "Higher values improve cache hit rates but consume more memory. "
+ + "Only effective when hfile.block.cache.enabled is true.");
+
+ public static final ConfigProperty<Integer>
METADATA_HFILE_BLOCK_CACHE_TTL_MINUTES = ConfigProperty
+ .key(METADATA_PREFIX + ".hfile.block.cache.ttl.minutes")
+ .defaultValue(60)
+ .markAdvanced()
+ .sinceVersion("1.1.0")
+ .withDocumentation("Time-to-live (TTL) in minutes for cached HFile
blocks. Blocks are evicted "
Review Comment:
This config is used in `expireAfterWrite` in the builder of Caffeine.
```
Caffeine.newBuilder()
.maximumSize(maxCacheSize)
.expireAfterWrite(Duration.ofMillis(timeUnit.toMillis(expireAfterWrite)))
.build();
```
I checked that it does not account for access pattern, so it will be evicted
out at t+60 min even though there is an access at t+59 min.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
