BlakeOrth commented on code in PR #19366: URL: https://github.com/apache/datafusion/pull/19366#discussion_r2628815473
########## docs/source/library-user-guide/upgrading.md: ########## @@ -45,15 +45,23 @@ directly on the `Field`. For example: In prior versions, `ListingTableProvider` would issue `LIST` commands to the underlying object store each time it needed to list files for a query. To improve performance, `ListingTableProvider` now caches the results of -`LIST` commands for the lifetime of the `ListingTableProvider` instance. +`LIST` commands for the lifetime of the `ListingTableProvider` instance or +until a cache entry expires. Note that by default the cache has no expiration time, so if files are added or removed from the underlying object store, the `ListingTableProvider` will not see those changes until the `ListingTableProvider` instance is dropped and recreated. -You will be able to configure the maximum cache size and cache expiration time via a configuration option: +You can configure the maximum cache size and cache entry expiration time via configuration options: -See <https://github.com/apache/datafusion/issues/19056> for more details. +`datafusion.runtime.list_files_cache_limit` +`datafusion.runtime.list_files_cache_ttl` Review Comment: `*` made `prettier`complain so I had to go for `- ` instead. In either case these are formatting better now, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
