paleolimbot commented on issue #12644: URL: https://github.com/apache/datafusion/issues/12644#issuecomment-3160925131
I agree that DataFusion would benefit from embracing its own definition of a (dynamic, session-registered) data type that allows a type author to specify behaviour in such a way that DataFusion internals (e.g., optimizer rules, casting) can benefit things that are not just built-in arrow types (and/or prevent DataFusion internals from introducing correctness issues that may arise from ignoring or dropping metadata). I'm also sensing resistance to take on responsibility for that, which is OK (engines that use DataFusion can implement all of this on top of metadata, although it is quite a bit of work that perhaps doesn't have to be duplicated if it's in DataFusion). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
