NoahKusaba opened a new pull request, #116:
URL: https://github.com/apache/datafusion-iceberg/pull/116

   ## Which issue does this PR close?
   
   - Closes #56.
   - Closes #57.
   - Part of #24.
   
   ## Rationale for this change
   
   `IcebergSchemaProvider` builds each table's provider once, when the catalog 
is registered, and every query reuses it with the schema the table had then. 
After another client changes the schema, INSERTs fail (#56) and reads return 
wrong results or panic (#57).
   
   ## What changes are included in this PR?
   
   - `IcebergSchemaProvider` keeps only table names. Each `table()` lookup 
loads the table and builds a new provider, including for `$` metadata tables.
   - Registering the catalog no longer loads every table.
   - Removes the now-unused `IcebergCatalogTableProvider::metadata_table`.
   
   Table names are still listed once, so tables other clients create or drop 
stay invisible (rest of #24).
   
   ## Are there any user-facing changes?
   
   Queries see schema changes made by other clients. Each lookup costs one 
`load_table`, on top of the one `scan`/`insert_into` already does; a follow-up 
can have the provider reuse the loaded table. No API changes.
   
   ## Are these changes tested?
   
   Yes. `tests/table_lookup.rs` has a session use a table, another client 
change its schema, and the session use it again. Both tests fail on `main` with 
the errors from the issues.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to