paleolimbot commented on issue #8730: URL: https://github.com/apache/arrow-rs/issues/8730#issuecomment-3773385937
> no longer allows separating physical arrays from extension types in the type system. DateTime types are in the DataType enum and integrate nicely with cast, pretty print, file readers and writers, etc. Perhaps these should have been implemented as extension types to better clearly separate responsibility but here we are 🙂. We removed all mention of "phsyical" and "logical" from the Arrow spec documentation for this reason. > `pub enum NativeOrExtensionArray` I think this is a totally reasonable solution, although I'm not sure arrow-rs is the home for it (as Raphael suggested). An `extendable-arrow` crate with a DataType/Field/Schema/Array/RecordBatch stack is certainly possible. We basically inline helpers for most of those in SedonaDB and can mostly do what we need to do with only a few ugly workarounds. From a DataFusion angle this still involves a lot of rewriting (which is not a problem as long as somebody is willing to do it, review it, and the downstream implications are acceptable to users of DataFusion). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
