rizaon commented on PR #4518: URL: https://github.com/apache/iceberg/pull/4518#issuecomment-1149113179
One way to avoid churn / pressure in cache is to detect what is the reason of most recent cache eviction. Say, if CachingFileIO tend to evict most entries in short time due to total memory size constraint, CachingFileIO should consider just bypassing cache for some period or until some entries evicted due to expiration rather than size limit. We can do that with caffeine cache by implementing RemovalListener https://www.javadoc.io/doc/com.github.ben-manes.caffeine/caffeine/2.3.1/com/github/benmanes/caffeine/cache/RemovalListener.html Another way is to impose high watermark limit on cache total weight. Thus, bypassing the cache once this high watermark limit exceeded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
