ion-elgreco commented on issue #823: URL: https://github.com/apache/datafusion-python/issues/823#issuecomment-2325325928
> For Table Provider, what I've been investigating is how we could do something like `register_table_provider`. But even scratching the surface of making this stable across versions is a deep rabbit hole. Suppose I *only* wanted to support `scan` in a Table Provider. I need to export a stable version of `Session`, `Expr`, and `ExecutionPlan`. But then `Session` and `ExecutionPlan` require even more. > > On the one hand, I can see wanting to just implement `register_delta_table` and add code specific to the python interface on delta-rs. That might be the fastest way to get us where we want to be but it's not the best long-term solution probably. A `register_delta_table` in Python Datafusion? That might bloat your binary since you include delta-rs and it will limit your Datafusion version. I think pola-rs has built their plugins around a C interface, haven't dived to deep in their internals yet -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
