alamb commented on code in PR #19616:
URL: https://github.com/apache/datafusion/pull/19616#discussion_r2661565794
##########
datafusion/core/src/execution/context/mod.rs:
##########
@@ -1327,12 +1329,34 @@ impl SessionContext {
&& table_provider.table_type() == table_type
{
schema.deregister_table(&table)?;
+ if table_type == TableType::Base {
+ self.drop_list_files_cache_entries(&table_ref,
&table_provider);
Review Comment:
This API seems pretty specific and it isn't ideal in my mind that
SessionContext now must check for a specific implementation -- I suggest a more
generic method to the `ListFilesCache` trait so we can keep this logic in the
implementation of `ListingTable`
https://github.com/apache/datafusion/blob/24e0983d4763c39cd5c6e7c414ae9d7629573736/datafusion/execution/src/cache/cache_manager.rs#L83-L82
Maybe something like
```rust
pub trait ListFilesCache {
...
/// Notifies the cache that a specific table has been dropped.
///
/// The table may clear up internal state that is specific to this table.
pub fn table_dropped(table_ref: &TableReference) -> Result<()>;
}
```
##########
datafusion/execution/src/cache/list_files_cache.rs:
##########
@@ -146,9 +149,12 @@ pub const DEFAULT_LIST_FILES_CACHE_MEMORY_LIMIT: usize =
1024 * 1024; // 1MiB
/// The default cache TTL for the [`DefaultListFilesCache`]
pub const DEFAULT_LIST_FILES_CACHE_TTL: Option<Duration> = None; // Infinite
+#[derive(PartialEq, Eq, Hash, Clone, Debug)]
+pub struct TableScopedPath(pub Option<TableReference>, pub Path);
Review Comment:
I would personally suggest using named fields here so readers don't have to
refer to the definition to know what `.0` and `.1` are -- so like
```rust
pub struct TableScopedPath {
pub table: Option<TableReference>,
pub path: Path
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
