naikaayush commented on code in PR #45:
URL: https://github.com/apache/datafusion-iceberg/pull/45#discussion_r4375458371
##########
crates/datafusion/src/schema.rs:
##########
@@ -173,38 +175,28 @@ impl SchemaProvider for IcebergSchemaProvider {
let tables = self.tables.clone();
let name_clone = name.clone();
- // Use tokio's spawn_blocking to handle the async work on a blocking
thread pool
- let result = tokio::task::spawn_blocking(move || {
- // Create a new runtime handle to execute the async work
- let rt = tokio::runtime::Handle::current();
- rt.block_on(async move {
- // Verify the input table is empty - CREATE TABLE only accepts
schema definition
- ensure_table_is_empty(&table).await?;
-
- catalog
- .create_table(&namespace, table_creation)
- .await
- .map_err(to_datafusion_error)?;
-
- // Create a new table provider using the catalog reference
- let table_provider = IcebergTableProvider::try_new(
- catalog.clone(),
- namespace.clone(),
- name_clone.clone(),
- )
- .await?;
+ block_on_catalog("create-table", async move {
Review Comment:
i dug through the original commit, design doc, and reviews, both methods
were introduced on `SchemaProvider` in eb305c5.
the initial implementation just inserted/removed providers in a map. i was
not able to find any discussion on why it is sync.
more interestingly, when `SchemaProvider::table()` became async in
[#4607](https://github.com/apache/datafusion/pull/4607), async registration was
spoken of, and tustvold said it could be added as a follow-up.
https://github.com/apache/datafusion/pull/4607#issuecomment-1371128793
so looks like async registration was deferred. but there is a separate
rationale for keeping query planning sync, but I don’t think that makes sense
for restrictions on async catalog mutations. this seems worth revisiting
upstream alongside [#19634](https://github.com/apache/datafusion/issues/19634)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
