mkleen opened a new pull request, #22613: URL: https://github.com/apache/datafusion/pull/22613
## Which issue does this PR close? - Closes https://github.com/issues/assigned?issue=apache%7Cdatafusion%7C22359. ## Rationale for this change This pr introduces a new cache which merges the functionality of file-metadata cache, list-files caches and file-statistics cache into one generic implementation. This removes a lot of redundant code. ## What changes are included in this PR? - Introduce a generic `DefaultCache` with a Cache trait and an implemenation with LRU eviction, memory accounting and ttl. - Migrate all cache tests to use the new `DefaultCache`. - Replace file-metadata cache, list-files-cache and file-statistics-cache implementations with the generic version. ## Are these changes tested? Yes, all previous tests are still present and passing. They had to be slighlty adapted because the new cache also takes the cache-key into account for memory accounting which wasn't the case for all previous implementations. The tests are still at the same location to have a diff for reviews. Once the tests are reviewed and approved they should probably move to the generic cache implemenation and the files `file_statistics_cache.rs`, `file_metadata_cache.rs` an `list_files_cache.rs` should be removed. ## Are there any user-facing changes? The traits `FileStatisticsCache`, `ListFilesCache` and `FileMetadataCache` are replaced with `Cache<TableScopedPath, CachedFileMetadata>`, `Cache<TableScopedPath, CachedFileList>` and `Cache<Path, CachedFileMetadataEntry>`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
