Hi everyone, We’ve been discussing the Modular Footer for quite a while, including through a dedicated footer working group meeting every two weeks, and we finally have a proposal ready for broader review.
The proposal covers the motivation, format design, compatibility considerations, and benchmark results. Huge thanks to everyone who has contributed to the project and the discussions so far. The proposal doc, parquet-format PR, and tracking issue are all available here: Proposal doc: https://docs.google.com/document/d/1xN4Nqh-fh-eH-cUz9e6Rqs_gPXRBKUh4s29EzX_NU8M/edit?usp=sharing PR: https://github.com/apache/parquet-format/pull/623/changes Parquet-format issue: https://github.com/apache/parquet-format/issues/622 The proposal introduces several significant conceptual changes, so I expect there will be a lot of discussion and feedback. For the first round of review, I’d suggest focusing primarily on the high-level direction and design choices, and leaving more detailed questions (for example, whether a particular field can be deprecated) for later discussion. I think this will help us converge on the major design questions first and keep the review process moving efficiently. Thanks! Best, Jiayi
