goldmedal commented on issue #12394: URL: https://github.com/apache/datafusion/issues/12394#issuecomment-2352110457
> We could potentially change build() to return an Arc but I don't know what implications that has > I guess it would be a huge breaking change for `SessionStateBuilder` and `SessionContext`. The main change is https://github.com/apache/datafusion/blob/7bd77477b3aa65d23fc096db140a08617ba2bd55/datafusion/core/src/execution/context/mod.rs#L351-L358 The `SessionStateBuilder` should produce an `RwLock` reference, and the `SessionContext` should be created with it. I'm not sure how many downstream projects would be impacted. > Maybe we could change the DynamicTableProvider to keep a reference to some part of the SessionState (rather than an Arc to the whole thing 🤔 ) but that might be messy I think the main challenge is URL resolution. The `DynamicListTableFactory` relies on `ListingTableConfig` to resolve the URL path and build the table https://github.com/apache/datafusion/blob/7bd77477b3aa65d23fc096db140a08617ba2bd55/datafusion/core/src/datasource/dynamic_file.rs#L71-L73 It’s not easy to extract only the required parts for this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
