tustvold commented on issue #7075: URL: https://github.com/apache/arrow-rs/issues/7075#issuecomment-2638073743
> Is there any chance for an ObjectStore C FFI to be on the roadmap? I will be frank, ObjectStore has a fairly large, async API, and so defining and maintaining an FFI interface for it would be a fairly substantial undertaking. Given what I know of the interests of the various maintainers, I think such an initiative would likely stand the best chance of success incubating as a third-party project. > I did float the idea of using libloading to the Polars devs but the main concern/blocker that was raised was the lack of a stable C FFI for ObjectStore IMO I would suggest adding support for custom ObjectStore within polars from the Rust API first, i.e. introducing something similar to [ObjectStoreRegistry](https://docs.rs/datafusion/latest/datafusion/datasource/object_store/trait.ObjectStoreRegistry.html). Once such an abstraction exists, then it will be possible to devise ways to potentially orchestrate that from python code. This wouldn't necessarily require a C FFI, for example, you could potentially build a shared library bundling both polars and your extension code, and then use that from python or something along those lines. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
