For context - an approach to extension types was proposed to Parquet in the past [1][2] but was met with opposition because it was somewhat duplicating logical types and not really reducing the cost of specifying types (consensus building).
I like Weston's proposal too, but I am not sure the Parquet community wants to deal with blessing individual extension types. What we could propose is that Parquet specifies how Arrow-compatible extension types serialize into Parquet metadata. This would enable third party extension types and if those show enough potential Parquet might canonicalize the popular ones. I have a vague plan to raise such a proposal soon. Regarding the ongoing vote in this thread and Mandy's proposal, should we put it on hold until we agree on this or continue with the vote? [1] https://github.com/apache/parquet-format/pull/451 [2] https://github.com/rok/parquet-format/pull/5 Best, Rok On Mon, Nov 9, 2026 at 10:08 AM Zehua Zou <[email protected]> wrote: > Hello, > > Should we initiate a discussion on the Parquet mailing list? > > Regards > > Zehua Zou > > > On Nov 9, 2026, at 16:22, Antoine Pitrou <[email protected]> wrote: > > > > > > Hello, > > > > I have roughly the same position as Weston here. > > > > Regards > > > > Antoine. > > > > > > > > Le 09/11/2026 à 06:04, Weston Pace a écrit : > >>> Are you saying there should be super-project canonical extension > types? :) > >> No. I am suggesting the Parquet PMC publish an extension-types.md > document > >> in [1] that describes what the extension type definitions are. This > would > >> be exactly like [2]. This lets the Parquet team bless whatever > >> unambiguous definition they like. Anyone wanting to consume custom > Parquet > >> types must use those definitions. The Parquet team is then in complete > >> control and doesn't need to seek the Arrow team's blessing. The Arrow > team > >> can then create a link back to that document in [3] in much the same > way we > >> do for GeoArrow. > >> For example, the Substrait type system does not name all Arrow types. > in > >> Arrow, we publish [4] which describes Substrait extension types to map > to > >> the existing Arrow types. > >> Or, as another example, Lance's blob type is an Arrow extension type > >> defined in [5]. Consumers of Lance custom types consult the Lance spec > for > >> those types' definitions. > >>> Similarily, I was thinking of adding more canonical Parquet extension > >>> types like GEOMETRY and GEOGRAPHY because I found it really tricky > >>> when implementing Iceberg V3 types in iceberg-cpp where GeoArrow type > >>> were not an option. > >> If Iceberg has its own set of types (separate from Parquet and GeoArrow) > >> then Iceberg can also maintain an extension-types.md document right? > >> [1] https://github.com/apache/parquet-format/ > >> [2] https://github.com/geoarrow/geoarrow/blob/main/extension-types.md > >> [3] > >> > https://arrow.apache.org/docs/format/CanonicalExtensions.html#community-extension-types > >> [4] > >> > https://github.com/apache/arrow/blob/main/format/substrait/extension_types.yaml > >> [5] > >> > https://github.com/lance-format/lance/blob/main/docs/src/format/table/blob.md#table-schema > >> On Sat, Nov 7, 2026 at 7:00 AM Rok Mihevc <[email protected]> wrote: > >>> Thanks for the discussion Weston and Gang! > >>> > >>>> I also somewhat disagree with this characterization. As long as the > >>>> producer and consumer agree on the type then there is no need for > every > >>>> middleman (or even Arrow itself) to be aware of it. That's the entire > >>> goal > >>>> of extension types. IPC streams, the C data interface, Arrow flight, > and > >>>> language bindings should all respect existing metadata and should > _not_ > >>>> strip it out, even if they don't recognize it. > >>> > >>> This is a great point and it's ideally how things should work. Now how > do > >>> we get there? Should Parquet readers that output Arrow map Parquet > logical > >>> types into "parquet.<logical_type>" extension types? If this requires > >>> non-trivial logic, to make the mapping unambiguous, are we implicitly > >>> specifying extension types at the conversion layer? > >>> > >>>> and so I will turn my vote into a +1 if needed for passage. Otherwise > >>> I'll > >>>> leave it as a half vote to discourage Arrow taking on the mantle of > >>>> blessing another format's types. > >>> > >>> Are you saying there should be super-project canonical extension > types? :) > >>> > >>> > >>> Best, > >>> Rok > >>> > >>> On Sat, Nov 7, 2026 at 2:57 AM Gang Wu <[email protected]> wrote: > >>> > >>>> Thanks Weston for raising this! I have a different opinion. > >>>> > >>>> Canonical extension type for Parquet types like VARIANT、FILE and > future > >>>> VECTOR brings better interoperability to different Parquet bindings in > >>>> Arrow. AFAIK, it would be a headache if different bindings define > their > >>>> own custom extensions then we end up with different interpretations of > >>>> ARROW:schema serialization across bindings. > >>>> > >>>> Similarily, I was thinking of adding more canonical Parquet extension > >>>> types like GEOMETRY and GEOGRAPHY because I found it really tricky > >>>> when implementing Iceberg V3 types in iceberg-cpp where GeoArrow type > >>>> were not an option. > >>>> > >>>> On Sat, Nov 7, 2026 at 6:38 AM Weston Pace <[email protected]> > >>> wrote: > >>>> > >>>>> +0.5 > >>>>> > >>>>> I guess I'm a bit late to the party since we already have > >>>>> arrow.parquet.variant but it feels like parquet-specific types would > >>> just > >>>>> fit the definition for community extension types[1]. In other words, > >>>>> Parquet is free to define whatever extension types they want and > there > >>> is > >>>>> no need for Arrow PMC approval in the same way that GeoArrow is free > to > >>>>> define whatever extension types they want. > >>>>> > >>>>>> To ensure full interoperability, Arrow requires a matching canonical > >>>>>> in-memory extension type so that file fields do not lose their > >>>>>> semantic metadata during serialization across IPC streams, the C > >>>>>> Data Interface, Arrow Flight, and language bindings. > >>>>> > >>>>> I also somewhat disagree with this characterization. As long as the > >>>>> producer and consumer agree on the type then there is no need for > every > >>>>> middleman (or even Arrow itself) to be aware of it. That's the > entire > >>>> goal > >>>>> of extension types. IPC streams, the C data interface, Arrow flight, > >>> and > >>>>> language bindings should all respect existing metadata and should > _not_ > >>>>> strip it out, even if they don't recognize it. > >>>>> > >>>>> That being said, I am happy for someone (even if it is Arrow and not > >>>>> Parquet) to bless a canonical extension type for any Parquet-specific > >>>> types > >>>>> and so I will turn my vote into a +1 if needed for passage. > Otherwise > >>>> I'll > >>>>> leave it as a half vote to discourage Arrow taking on the mantle of > >>>>> blessing another format's types. > >>>>> > >>>>> [1] > >>>>> > >>>>> > >>>> > >>> > https://arrow.apache.org/docs/format/CanonicalExtensions.html#community-extension-types > >>>>> > >>>>> On Fri, Nov 6, 2026 at 11:36 AM Mandukhai Alimaa < > >>>>> [email protected]> > >>>>> wrote: > >>>>> > >>>>>> Hi everyone, > >>>>>> > >>>>>> The FILE column type is being adopted across Parquet, Delta Lake, > >>>>>> Apache Iceberg, and Apache Spark so the broader ecosystem can > >>> natively > >>>>>> leverage structured file-level references. To ensure full > >>>>>> interoperability, Arrow requires a matching canonical in-memory > >>>>>> extension type so that file fields do not lose their semantic > >>> metadata > >>>>>> during serialization across IPC streams, the C Data Interface, Arrow > >>>>>> Flight, and language bindings. > >>>>>> > >>>>>> Following up on the discussion thread [1] and the subsequent format > >>>>>> change PR [2], we (@rok and I) would like to propose a formal vote > to > >>>>>> add arrow.parquet.file as a canonical extension type to the Apache > >>>>>> Arrow format specification. This proposal already had reviews from > >>>>>> four different committers on GitHub and closely matches the Parquet > >>>>>> file logical type. > >>>>>> > >>>>>> [1] > >>>>> > https://lists.apache.org/thread.html/5qw2k7rwstk0ol9ymds5qym81myznmpl > >>>>>> [2] https://github.com/apache/arrow/pull/51248 > >>>>>> [3] GitHub Issue: https://github.com/apache/arrow/issues/51143 > >>>>>> > >>>>>> The vote will be open for at least 72 hours. > >>>>>> > >>>>>> Please cast your vote: > >>>>>> [ ] +1 Accept this proposal > >>>>>> [ ] +0 > >>>>>> [ ] -1 Do not accept this proposal because... > >>>>>> > >>>>>> Thanks > >>>>>> > >>>>> > >>>> > >>> > > > >
