dutyu commented on code in PR #18102:
URL: https://github.com/apache/doris/pull/18102#discussion_r1148284620
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java:
##########
@@ -120,14 +121,22 @@ public HivePartition load(PartitionCacheKey key) throws
Exception {
}
}, executor));
- fileCache =
CacheBuilder.newBuilder().maximumSize(Config.max_external_file_cache_num)
-
.expireAfterAccess(Config.external_cache_expire_time_minutes_after_access,
TimeUnit.MINUTES)
- .build(CacheLoader.asyncReloading(new
CacheLoader<FileCacheKey, ImmutableList<InputSplit>>() {
- @Override
- public ImmutableList<InputSplit> load(FileCacheKey key)
throws Exception {
- return loadFiles(key);
- }
- }, executor));
+ // if the file.meta.cache.ttl-second is equal or greater than 0, the
cache expired will be set to that value
+ int fileMetaCacheTtlSecond = NumberUtils.toInt(
+
(catalog.getProperties().getOrDefault(HMSExternalCatalog.FILE_META_CACHE_TTL_SECOND,
"")), -1);
+
+ CacheBuilder<Object, Object> fileCacheBuilder =
CacheBuilder.newBuilder().maximumSize(Config.max_external_file_cache_num);
+ if (fileMetaCacheTtlSecond >= 0) {
+ fileCacheBuilder.expireAfterWrite(fileMetaCacheTtlSecond,
TimeUnit.SECONDS);
+ }
Review Comment:
> We also need to let user disable the file cache. Maybe we can use `0` as a
special value? `0` means file cache is disabled. `>0` means file cache with ttl
`-1` means file cache no ttl
I think maybe we can make this logic easier, `file.meta.cache.ttl-second`
should just be greater or equal to 0, 0 means disable the file cache, >0 means
enable ttl with file cache, other values are illegal，if user does not want to
enable the ttl with file cache, just does not set `file.meta.cache.ttl-second`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
