JanKaul commented on issue #3777: URL: https://github.com/apache/arrow-datafusion/issues/3777#issuecomment-1309819841
> Hi @JanKaul and @v0y4g3r -- I wonder if the (very) newly added `TableProviderFactories` may be helpful / fulfill this usecase #4126 Hi, thanks for the quick reply. I'm trying to implement an [iceberg](https://iceberg.apache.org/) catalog for datafusion. It functions much like a common catalog with schemas(namespaces) and table identifiers. From what I understood about the `TableProviderFactory`, it wouldn't really fit the use case. One could use the `url` field to pass in a table identifer like "my_catalog.my_schema.my_table" and get the `TableProvider` from that. But I don't think this is what it is intended for. > I found a workaround. In this case you can spawn a thread dedicating to block: > > ```rust > use std::collections::HashMap; > use std::sync::Arc; > use tokio::sync::RwLock; > > trait CatalogList { > fn catalog(&self, name: &str) -> Option<Arc<CatalogProvider>>; > } > > struct CatalogProvider; > > #[derive(Default)] > struct CatalogListImpl { > map: Arc<RwLock<HashMap<String, Arc<CatalogProvider>>>>, > } > > impl CatalogList for CatalogListImpl { > fn catalog(&self, name: &str) -> Option<Arc<CatalogProvider>> { > let map = self.map.clone(); > let name = name.to_string(); > std::thread::spawn(move || { > futures::executor::block_on(async { > map.read().await.get(&name).cloned() > }) > }).join().unwrap() > } > } > > #[tokio::main] > async fn main() { > let catalog_list = CatalogListImpl::default(); > assert!(catalog_list.catalog("hello").is_none()); > } > ``` Awesome, thanks for the heads up. It is probably not the cleanest thing to span another thread while the rest of the application is executed with tokio, but it should work for the time being. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
