timsaucer commented on issue #823: URL: https://github.com/apache/datafusion-python/issues/823#issuecomment-2326273866
No, sorry, I wasn't clear in what I was suggesting. I was wondering if we should expose something along the lines of `register_delta_table` and then provide a FFI stable interface that narrowly supports the needs of delta-rs. As I'm diving in deeper I'm discovering more difficulties. I think this topic is complex. Since different versions of the rust compiler and different versions of datafusion would all lead to different binary layouts, it's really not as simple as exposing `register_table_provider`. One way around that *could* be to create a compiler dependency for any official wheels - but then we're adding a new requirement on delta-rs which seems fragile long term, especially if you want to update rustc versions (or we do and you want to keep them where they are). Lowest level effort to get this up and running would be the last idea. *Maybe* that's okay? I'd hate to add that kind of build dependency. Also anyone who is developing would have to make sure they either use the same *or* build both wheels locally. My thoughts are a little jumbled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
