GitHub user CurtHagenlocher added a comment to the discussion: Proposal: ADBC file drivers with pluggable storage
Some random thoughts in no particular order: Whether it's under an ADBC umbrella or not, this feels like it would be a really useful bit of infrastructure to have. Maybe it makes more sense for it to be an Arrow thing, and I almost started there instead of here. After all, I could have expressed this as the desire for a C ABI for Arrow Filesystems, and this fits fairly naturally into a previously-stated desire to be able to trim down the size of PyArrow. I will admit a certain obsession with Substrait. It's clearly aimed at solving problems that I have, even if it doesn't (yet) do so in ways that quite work for me today. The only competition it has on the horizon is Viktor Leis' "SaneIR" and AFAIK there haven't been any real details released publicly. That said, I wasn't going to mention it in this proposal but the AIs seem to have learned my preferences and snuck it in and I decided not to edit it out. I do think there's something valuable here and I hope to spend more time on it in the near future. I'm not strongly tied to any particular approach or terminology. "Driver" felt natural because of the dynamically loadable connotations and because there are ODBC and JDBC drivers for file sources like Excel. I'm not super keen on "data assets" but I don't think the success or failure of this spec is strongly tied to the names we pick and am very sympathetic to the pain of having to describe how yes, this is a driver and that is a driver but they're different kinds of drivers. GitHub link: https://github.com/apache/arrow-adbc/discussions/4877#discussioncomment-18928121 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
