alamb commented on code in PR #17031:
URL: https://github.com/apache/datafusion/pull/17031#discussion_r2255229675
##########
datafusion/execution/src/cache/cache_manager.rs:
##########
@@ -102,12 +118,19 @@ impl CacheManager {
}
/// Get the file embedded metadata cache.
- pub fn get_file_metadata_cache(&self) -> Option<Arc<dyn
FileMetadataCache>> {
- self.file_metadata_cache.clone()
+ pub fn get_file_metadata_cache(&self) -> Arc<dyn FileMetadataCache> {
+ Arc::clone(&self.file_metadata_cache)
+ }
+
+ /// Get the limit of the file embedded metadata cache.
+ pub fn get_file_metadata_cache_limit(&self) -> Option<usize> {
+ self.file_metadata_cache.cache_limit()
}
}
-#[derive(Clone, Default)]
+const DEFAULT_FILE_METADATA_CACHE_LIMIT: usize = 1024 * 1024 * 1024; // 1G
Review Comment:
I think 1GB is too high. I think should set it to something more like 50MB
or 100MB and ideally make it easy to tell when that limit was exceeded (some
sort of metrics or statistics)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
