voonhous commented on code in PR #13724:
URL: https://github.com/apache/hudi/pull/13724#discussion_r2335996012
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -527,6 +527,33 @@ public final class HoodieMetadataConfig extends
HoodieConfig {
.withDocumentation("Max size in MB below which metadata file (HFile)
will be downloaded "
+ "and cached entirely for the HFileReader.");
+ public static final ConfigProperty<Boolean>
METADATA_HFILE_BLOCK_CACHE_ENABLED = ConfigProperty
+ .key(METADATA_PREFIX + ".hfile.block.cache.enabled")
+ .defaultValue(false)
+ .markAdvanced()
+ .sinceVersion("1.1.0")
+ .withDocumentation("Enable HFile block-level caching for metadata files.
This caches frequently "
+ + "accessed HFile blocks in memory to reduce I/O operations during
metadata queries. "
+ + "Improves performance for workloads with repeated metadata access
patterns.");
+
+ public static final ConfigProperty<Integer> METADATA_HFILE_BLOCK_CACHE_SIZE
= ConfigProperty
+ .key(METADATA_PREFIX + ".hfile.block.cache.size")
+ .defaultValue(100)
+ .markAdvanced()
+ .sinceVersion("1.1.0")
+ .withDocumentation("Maximum number of HFile blocks to cache in memory
per metadata file reader. "
+ + "Higher values improve cache hit rates but consume more memory. "
+ + "Only effective when hfile.block.cache.enabled is true.");
+
+ public static final ConfigProperty<Integer>
METADATA_HFILE_BLOCK_CACHE_TTL_MINUTES = ConfigProperty
+ .key(METADATA_PREFIX + ".hfile.block.cache.ttl.minutes")
+ .defaultValue(60)
+ .markAdvanced()
+ .sinceVersion("1.1.0")
+ .withDocumentation("Time-to-live (TTL) in minutes for cached HFile
blocks. Blocks are evicted "
+ + "from the cache after this duration to prevent memory leaks. "
+ + "Only effective when hfile.block.cache.enabled is true.");
Review Comment:
Make sense, will move this to `HoodieReaderConfig`. Then remove
`HFileReaderConfig` entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
