gszadovszky commented on pull request #798: URL: https://github.com/apache/parquet-mr/pull/798#issuecomment-650992678
@maccamlc, The main problem I think is that the spec does not say anything about how the thrift objects shall be used. The specification is about the semantics of the schema and it is described using the parquet schema _language_. But, in the file there is no such _language_, we only have [thrift objects](https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift). When the specification says something about the _logical types_ (e.g. `MAP`) it does not say anything about which thrift structure should be used (the converted type [`MAP`](https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L53) or the logical type [`MAP`](https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L324)). We added the new logical type structures in the thrift to support enhanced ways to specify _logical types_ (e.g. [`TimeStampType`](https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L272)). The idea for backward compatibility was to write the old converted types wherever it make sense (the semantics of the actual _logical type_ is the same as was before) along with the new logical type structures. So, related to `MAP_KEY_VALUE`, I think, we shall write it at the correct place if it was written before (prior to `1.11.0`) and it helps for other readers but do not expect it to be there. Cheers, Gabor ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
