This is a very good point, Antoine! Stepping back, Parquet’s openness comes from ubiquitous interoperability through time and vendor-space, not just from an open specification.
The current approach seems to work: don’t write features your readers don’t support; readers fail only when an unsupported feature is on the critical path and ignore what they don’t need. I fear versioning might not improve interoperability but would fragment Parquet into incompatible dialects. And the end result - that a user encounters a file with data they cannot read - stays the same whichever way? regards, Will On Thu, 3 Sept 2026 at 08:58, Antoine Pitrou <[email protected]> wrote: > Le 03/09/2026 à 01:08, Andrew Lamb a écrit : > > I am not sure it is proof, but here is my logic: > > > > Given any particular Parquet file, there is no simple, practical way to > > understand what versions of what products/projects can read it. > > > > I understand we have the compatibility chart[1] (which I helped to > create), > > but that doesn't have entries for end user systems (e.g. Spark version > 3.5) > > and I believe there is a widely held belief that writers must stick to > very > > conservative feature lists (see blogs below) > > > > In my mind, having a single clear version in the file (this file is > written > > with version X Parquet features), makes it clear, unambiguous, and easy > to > > verify what systems can read it (e.g. DB X can read parquet files with > > version Y). > > My underlying question is: why is it easier if the version number is > written in the file, than simply by trying to read the file? > > (you have to read the file anyway to get the version number that's > written in it :-)) > > Regards > > Antoine. > > >
