Thanks Aihua! The geo type implementation in Iceberg is currently blocked by this release. Really looking forward to it.
Jia On Wed, Jul 16, 2025 at 10:47 PM Gábor Szádovszky <ga...@apache.org> wrote: > My concern was related to the current stage of the Variant specification > and the fact that we started talking about releasing parquet-java with > Variant features. > If we formally release parquet-format with the finalized Variant spec > first, then I have no concerns about writing Variant values in the upcoming > parquet-java release. Otherwise, we need to block it by default and mark it > as an experimental feature. > > Cheers, > Gabor > > Aihua Xu <aihu...@gmail.com> ezt írta (időpont: 2025. júl. 16., Sze, > 19:37): > > > Hi Gabor and all, > > > > Here’s my current understanding of the progress on the *Variant* support > in > > Parquet: > > > > - > > > > Per Parquet's requirements, we need at least two reference > > implementations to finalize the Variant logical type specification. > > - > > > > The community is actively working on Java, Go, and Rust > implementations: > > - > > > > Java already has the encoding and shredding implementations in > place: > > - > > > > Variant Decoding < > > https://github.com/apache/parquet-java/pull/3197> > > - > > > > Variant Encoding < > > https://github.com/apache/parquet-java/pull/3202> > > - > > > > Variant Shredding Writer > > <https://github.com/apache/parquet-java/issues/3223> > > - > > > > Variant Shredding Reader > > <https://github.com/apache/parquet-java/issues/3211> > > - > > > > Go also includes encoding and shredding support: > > - > > > > Variant Encoding/Decoding > > <https://github.com/apache/arrow-go/pull/344> > > - > > > > Variant Shredding <https://github.com/apache/arrow-go/pull/434> > > - > > > > Rust is currently working on the shredding implementation. > > > > In addition to these, we already have a full Variant implementation in > > Apache Iceberg, as well as in some closed-source engines. > > > > At this point, I’d like to check if we have enough implementation > coverage > > to move forward with finalizing the Variant spec. Would it make sense to > > start a vote thread at this stage? > > > > Ultimately, our goal is to release a new version of parquet-format and > > parquet-java that includes the Variant logical type, so that Iceberg and > > other engines can officially depend on it and proceed with further > > implementation. > > > > Let me know your thoughts and how we should proceed. > > > > Thanks, > > > > Aihua > > > > On Sun, Jul 13, 2025 at 10:08 PM Gábor Szádovszky <ga...@apache.org> > > wrote: > > > > > Hi, > > > > > > I was not able to open the recordings of the last meeting because of > > > permission issues. (Shouldn't these be accessible for anyone?) > > > So, I'm not sure if you have talked about this, but the Variant spec is > > > still not final. Since parquet-java already has Variant support, how do > > we > > > prevent writing potentially invalid Variant data with the proper > logical > > > types we will use for the finalized spec? Is it behind a feature flag? > > > > > > Cheers, > > > Gabor > > > > > > Aihua Xu <aihu...@gmail.com> ezt írta (időpont: 2025. júl. 11., P, > > 19:33): > > > > > > > Hi community, > > > > > > > > As discussed in the last community sync-up meeting, I'd like to > proceed > > > > with releasing *Parquet-Java 1.16.0*, which will include support for > > > > *geo-type* and *variant*. > > > > > > > > Please let me know if you have any objections or if you have any > > upcoming > > > > changes you'd like to include in this release. > > > > Thanks, > > > > Aihua > > > > > > > > > >
