alamb commented on issue #13838: URL: https://github.com/apache/datafusion/issues/13838#issuecomment-2554943551
> We are lucky to be doing engineering in a field where most problems of this type has workarounds and solutions, but when they start piling on, IMO it is a good signal that some lower-level design was wrong. In this case, it seems like that is the non-uniformity on the source side. So my "gut" feeling is that this change would basically push complexity around (make implementing `TableProviders` outside the DataFusion core more complicated) but I don't think I have much more than that opinion to share One potential way to proceed with this idea would be to sketch out what this idea would look like in a PR and try to adapt some existing open source table providers and see the impact These are some obvious candidates: - https://github.com/delta-io/delta-rs - https://github.com/datafusion-contrib/datafusion-table-providers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
