paleolimbot commented on issue #7063: URL: https://github.com/apache/arrow-rs/issues/7063#issuecomment-2805716506
> extension types automatically "just work" This is definitely true for some operations (e.g., arrow-select), but for others (e.g., cast, parse, print, write to CSV, arithmetic) it is very easy to do the wrong thing because there is no built-in alternative to capture that context: the `DataType` in its current form is not really "how the values of an array should be interpreted", but more like `ArrowStorageType` or `PhysicalTypeButAlsoTimestamps`. You are absolutely right that adding `Extension` to the `DataType` was never the intention of the original `DataType`...I think it is also true that a lot of usage (including DataFusion and the higher-level arrow crates) is not consistent with that intention. We have an internal workaround that we're pursuing for now, but I'll continue to play with/review alternatives both here and in DataFusion...if nothing else, it's a great way to get to know arrow-rs! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
