github-actions[bot] commented on code in PR #64705:
URL: https://github.com/apache/doris/pull/64705#discussion_r3466319270
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCacheEntry.java:
##########
@@ -155,31 +194,101 @@ public MetaCacheEntryStats stats() {
cacheStats.hitCount(),
cacheStats.missCount(),
cacheStats.hitRate(),
- cacheStats.loadSuccessCount(),
- cacheStats.loadFailureCount(),
- cacheStats.totalLoadTime(),
- cacheStats.averageLoadPenalty(),
+ successCount,
+ failureCount,
+ totalLoadTime,
+ totalLoadCount == 0 ? 0D : (double) totalLoadTime /
totalLoadCount,
cacheStats.evictionCount(),
invalidateCount.get(),
lastLoadSuccessTimeMs.get(),
lastLoadFailureTimeMs.get(),
lastError.get());
}
+ // Read the config dynamically so existing cache entries follow runtime
config updates.
+ private boolean isManualMissLoadEnabled() {
+ return Config.enable_external_meta_cache_manual_miss_load;
+ }
+
+ // Execute slow miss loads outside Caffeine's sync load path and suppress
stale write-back after invalidation.
+ private V getWithManualLoad(K key, Function<K, V> loadFunction) {
+ if (!effectiveEnabled) {
+ // Bypass cache entirely when the entry is disabled so manual miss
load does not relax disable semantics.
+ return loadAndTrack(key, loadFunction);
+ }
+
+ V value = data.getIfPresent(key);
+ if (value != null) {
+ return value;
+ }
+
+ synchronized (loadLock(key)) {
+ value = data.asMap().get(key);
+ if (value != null) {
+ return value;
+ }
+
+ long generation = invalidateGeneration.get();
+ V loaded = loadAndTrack(key, loadFunction);
+ if (generation != invalidateGeneration.get()) {
+ return loaded;
+ }
+
+ // Keep null results uncached so manual miss load matches
LoadingCache null-return behavior.
+ if (loaded == null) {
+ return null;
+ }
+
+ // Leave a narrow hook for tests to pause exactly before the cache
put race window.
+ beforeManualCachePutForTest(key, loaded);
+ data.put(key, loaded);
+ if (generation != invalidateGeneration.get()) {
Review Comment:
Even with the generation check below, this still briefly publishes a value
that was loaded before an invalidation. One interleaving is: a miss records
generation 0 and finishes the slow loader; `REFRESH CATALOG`/`invalidateKey`
increments the generation to 1 and returns; this thread then executes
`data.put(key, loaded)` and only removes it on the next lines; a third `get`
can hit `data.getIfPresent` before that removal and return the stale
schema/metadata after the refresh. Please avoid publishing values loaded under
an older generation, or attach/check the generation on cached values so readers
cannot return entries from a previous invalidation generation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
