paleolimbot commented on issue #7982: URL: https://github.com/apache/arrow-rs/issues/7982#issuecomment-4199947842
Just updating this while the details are fresh in my head from https://github.com/apache/arrow-nanoarrow/pull/861 : The IPC flatbuffers format doesn't allow for field metadata on the dictionary value type, which I'm guessing why it never made it to the DataType enum. Arrow C++ handles this ambiguity at IPC deserialization time via the extension registry (if the extension type rejects Dictionary as a storage type, value type gets the extension metadata instead and is then deserialized). Functionally there are no extension types that actually support dictionary storage (except maybe arrow.opaque, which can have any storage type but is usually binary), so the value type is almost always the thing that should have the extension. metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
