tustvold commented on PR #39200: URL: https://github.com/apache/arrow/pull/39200#issuecomment-1852881651
FWIW the design in arrow-rs currently is that extension types *only* exist in the metadata. The rationale being that by default kernels should treat them as the storage type. Imo it goes against the whole purpose of extension types if the only way to support them is for every kernel to be aware of all supported extension types. That's not to say there can't be special kernels for certain extension types, potentially selected as part of a query engine's planning, but I can't help feeling if an array can't be correctly treated as its physical type, something is a bit off. That's all to say I agree with coercing extension arrays to their physical types, arrow-rs goes even further by not having an extension array at all. That all being said, that is just my view on this, and I suspect others will feel differently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
