Hello,
I haven't read everything in detail yet, but I'm going to say upfront that I'm -1 on limiting sizes to 32 bits rather than the current 64 bits, unless it brings really sizable benefits (which I doubt, given the affected fields). Regards Antoine. On Thu, 11 Sep 2025 08:41:34 +0200 Alkis Evlogimenos <alkis.evlogime...@databricks.com.INVALID> wrote: > Hi all. I am sharing as a separate thread the proposal for the footer > change we have been working on: > https://docs.google.com/document/d/1kZS_DM_J8n6NKff3vDQPD1Y4xyDdRceYFANUE0bOfb0/edit > . > > The proposal outlines the technical aspects of the design and the > experimental results of shadow testing this in production workloads. I > would like to discuss the proposal's most salient points in the next sync: > 1. the use of flatbuffers as footer serialization format > 2. the additional limitations imposed on parquet files (row group size > limit, row group max num row limit) > > I would prefer comments on the google doc to facilitate async discussion. > > Thank you, >
