etseidl commented on PR #581: URL: https://github.com/apache/parquet-format/pull/581#issuecomment-4660826055
My issue with this (and #582) is that the version info (here the minor, there both major and minor), are encoded _after_ the entirety of the metadata. Readers will have to decode the entire footer to obtain adequate versioning info. The current version is an `i32`, why waste that on a single digit? We can encode a year based version as a decimal integer `202606`. Or if we want SemVer something like `2013000` (for `2.13.0`). Or, for just major/minor we could split the `i32` into two `i16`s (`0x2000D`). The advantage to keeping the current version field is that it will be the first thing encoded. You don't need a custom thrift decoder to examine a single VLQ encoded integer. Readers can then check the version _before_ passing the footer bytes on to whatever thrift decoder they're using. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
