CTTY commented on code in PR #1600:
URL: https://github.com/apache/iceberg-rust/pull/1600#discussion_r2277539503
##########
crates/integrations/datafusion/src/table/mod.rs:
##########
@@ -140,8 +152,15 @@ impl TableProvider for IcebergTableProvider {
filters: &[Expr],
_limit: Option<usize>,
) -> DFResult<Arc<dyn ExecutionPlan>> {
+ // Refresh table if catalog is available
+ let table = if let Some(catalog) = &self.catalog {
+
catalog.load_table(self.table.identifier()).await.map_err(to_datafusion_error)?
+ } else {
+ self.table.clone()
+ };
+
Review Comment:
This makes me wonder if storing table directly in the `IcebergTableProvider`
is correct... We could get a stale table if the provider doesn't have a catalog
table.
Iceberg-java has a
[`refresh()`](https://github.com/apache/iceberg/blob/7477f8bfbcf9cded7129bbf8ca7e91bcaa81c1ae/api/src/main/java/org/apache/iceberg/Table.java#L42)
interface which uses `TableOperation` to refresh metadata. In iceberg-rs we
don't have `TableOperation` and need to rely on catalog to refresh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
