Hi Antoine, > I don't understand what problem this is solving that is not already > solved, as readers are already able to emit a fine-grained error message > when they encounter an unsupported feature.
There are several proposals that are structural changes, that while you might get fine-grained error messages, they won't necessarily be pretty or understandable: 1. Stop writing path_in_schema (this ends up being a thrift parse error). 2. Non-contiguous pages (the initial proposal was to put -1 for some fields). 3. Move to a different footer layout (again a thrift parse-error). All of these would have error messages that would state the file is invalid but not in a way that is really meaningful to users (or at least hard to research until enough questions are asked online about what they indicate). (it was added in https://github.com/apache/arrow/pull/7502 and you > probably remember who the author was :-D) ' > I'd like to say there are mitigating factors here but let's say I'm at least consistent in approach and actually trying to do it properly for parquet :) On Thu, Jun 25, 2026 at 6:12 AM Antoine Pitrou <[email protected]> wrote: > > Hi Micah, > > Le 24/06/2026 à 19:50, Micah Kornfield a écrit : > > However, I think the > > version number should be treated as informational only (i.e. it shouldn't > > be validated by readers) and having a fine grained enumeration of > features > > a reader needs to understand is a better approach for the following > reasons: > > > > 1. It allows for readers to produce finer grained error messages for the > > options it doesn't support (for the 1 time cost of changing the magic > > number, which will produce "not a parquet file") > > I don't understand what problem this is solving that is not already > solved, as readers are already able to emit a fine-grained error message > when they encounter an unsupported feature. > > As an additional data point, we already added something like this in > Arrow and I don't know of an implementation that uses it: > > https://github.com/apache/arrow/blob/cffe22d6fe7f73836d2d9177b4b4fb2d1fcf4d1f/format/Schema.fbs#L54-L81 > > (it was added in https://github.com/apache/arrow/pull/7502 and you > probably remember who the author was :-D) > > Regards > > Antoine. > >
