DerGut commented on code in PR #3000:
URL: https://github.com/apache/iceberg-rust/pull/3000#discussion_r3799689323
##########
crates/integrations/datafusion/src/lib.rs:
##########
@@ -15,16 +15,36 @@
// specific language governing permissions and limitations
// under the License.
-mod catalog;
-pub use catalog::*;
-
mod error;
pub use error::*;
+mod catalog_access;
+mod catalog_provider;
+pub use catalog_provider::*;
pub mod physical_plan;
-mod schema;
+mod schema_provider;
pub mod table;
pub use table::table_provider_factory::IcebergTableProviderFactory;
pub use table::*;
pub(crate) mod task_writer;
+
+use std::fmt;
+
+use datafusion::catalog::Session as DFSession;
+use datafusion::error::Result as DFResult;
+use iceberg::SessionContext;
+
+/// Resolves an Iceberg [`SessionContext`] from a DataFusion session.
+///
+/// The DataFusion integration calls the resolver once while planning each
+/// session-aware scan or insert. The returned context is bound to that
+/// operation and, for inserts, is retained through transaction commit.
+/// Implementations should therefore return a stable Iceberg session identity
+/// for repeated operations from the same DataFusion session.
+pub trait SessionContextResolver: fmt::Debug + Send + Sync {
Review Comment:
Thanks again for the quick review!
Essentially, the `SessionContextResolver` is an adapter between the query
engine and Iceberg. The `SessionCatalog` is pure Iceberg and required to be
provided. The `SessionContextResolver` is specific to how Datafusion works (it
accepts a `datafusion::catalog::Session`) and cannot easily be abstracted
further and tied to the `SessionCatalog`.
In the Java implementation, we don't have an equivalent because its only
query engine integration (in the `apache/iceberg` repo at least) is Spark which
is single-tenant and doesn't need/ use a `SessionCatalog`.
A better example in the Java world would be the Trino-Iceberg connector,
which defines its own equivalent of the `SessionContextResolver` -> included in
the
[`TrinoRestCatalog`](https://github.com/trinodb/trino/blob/3a86d17b9457d9e01d9d92f3d8343bf6a3792c8e/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/rest/TrinoRestCatalog.java#L927).
Note that Trino is different in the sense that a Trino `ConnectorSession` is
already structured, and so a default implementation can be provided. This is
very different in the Datafusion world, where users provide their custom types
(more on this in the PR description).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
