alamb commented on code in PR #7620:
URL: https://github.com/apache/arrow-datafusion/pull/7620#discussion_r1349235611
##########
datafusion/execution/src/cache/cache_manager.rs:
##########
@@ -61,6 +79,10 @@ pub struct CacheManagerConfig {
/// Avoid get same file statistics repeatedly in same datafusion session.
/// Default is disable. Fow now only supports Parquet files.
pub table_files_statistics_cache: Option<FileStatisticsCache>,
+ /// Enable cache of files statistics when listing files.
+ /// Avoid get same files meta under same path repeatedly in same
datafusion session.
+ /// Default is disable.
Review Comment:
```suggestion
/// Enable cache of file metadata when listing files.
/// This setting avoids listing file meta of the same path repeatedly
/// in same session, which may be expensive in certain situations (e.g.
remote object storage).
/// Note that if this option is enabled, DataFusion will not see any
updates to the underlying
/// location.
/// Default is disable.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
