rdblue commented on issue #1578: URL: https://github.com/apache/iceberg/issues/1578#issuecomment-706620197
Maps with non-string keys are not allowed in Avro, but they are valid in Iceberg. That means we need to serialize `m: map<long, string>` differently. The approach we take is to store a map as a list of key/value pairs, just like you would get by calling `entrySet`. The list is needed because the map can have any number of key/value pairs. Omitting it would be equivalent to `m: struct<key: long, value: string>`. The Iceberg readers are built to handle this and reconstruct the map correctly on read. And we use a logical type annotation, `"logicalType": "map"`, to indicate to regular Avro readers that the `list<struct<long, string>>` that is decoded should be converted to `map<long, string>`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
