pitrou commented on code in PR #250: URL: https://github.com/apache/parquet-format/pull/250#discussion_r1617739978
########## README.md: ########## @@ -118,6 +118,65 @@ chunks they are interested in. The columns chunks should then be read sequentia  + ### PAR3 File Footers + + PAR3 file footer footer format designed to better support wider-schemas and more control + over the various footer size vs compute trade-offs. Its format is as follows: + - Data pages containing serialized Thrift metadata objects that were modeled as lists + in PAR1.These are stored contiguously with offsets stored in the FileMetadata. See + parquet.thrift for more details on each. + - Serialized Thrift FileMetadata Structure + - (Optional) 4 byte CRC32 of the serialized Thrift FileMetadata. + - 4-byte length in bytes (little endian) of the serialized FileMetadata structure. + - 4-byte length in bytes (little endian) of all preceding elements in the footer. + - 1 byte flag field to indicate features that require special parsing of the footer. Review Comment: Why limit ourselves while it would cost basically nothing to go for 64 bits from the start? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
