Hi folks, Should we resolve https://github.com/apache/parquet-java/pull/3393#issuecomment-5105352662 before releasing parquet-java? My understanding of the current state is that emitting nan_counts but using TYPE_DEFINED_ORDER could result in legacy readers seeing finite min/max stats and ignoring nan_counts. In this case, such readers could use the min/max stats for pruning and get wrong results. However, using the new IEEE sort order has its own risks as there are likely legacy readers that cannot parse unknown Thrift enum values.
This was also discussed in today's Parquet sync meeting and I will start a separate thread to discuss the right way forward here. Right now, it seems we have a disconnect between parquet-format, where adding the new sort order was considered a forward-compatible change, and reference implementations, where we seem to think that the new orders can only be used in the next major version bump. -- Div On Sun, Jul 26, 2026 at 11:45 PM Kevin Liu <[email protected]> wrote: > Here's a tracking issue for all the LICENSE / NOTICE fixes: "Ensure all > published JARs contain correct LICENSE and NOTICE files" #3687 ( > https://github.com/apache/parquet-java/issues/3687) > > This will > - include LICENSE and NOTICE files in every jar > - add CI to validate LICENSE and NOTICE files are included in every jar > - update LICENSE and NOTICE for jars that bundle 3rd party code (this is an > ASF release requirement) > > I just went through the process to update LICENSE and NOTICE files for > iceberg-java, so I'm still fairly familiar with the process. I would love > to have someone double check though :) > > > > > On Sun, Jul 26, 2026 at 1:13 PM Fokko Driesprong <[email protected]> wrote: > > > Thanks Kevin, let me look into that. > > > > I also found a behavioral change, which I believe is problematic. PR 3575 > > changes how we emit statistics: > > https://github.com/apache/parquet-java/pull/3575. When there is one or > > more row-groups that don't have bounds we don't produce statistics at the > > file level. The PR changes this by always emitting the null-values (which > > will automatically produce file-level stats). In Iceberg we rely on the > > fact when statistics are null when one or more row-groups don't have > stats: > > https://github.com/apache/iceberg/pull/17321#discussion_r3641772062. I > > think this behavioral change cause downstream issues which I think we > > should avoid as it might introduce correctness issues. > > > > I took the liberty of creating a PR to revert that specific change: > > https://github.com/apache/parquet-java/pull/3688. Please chime in to let > > me know your thoughts. > > > > Kind regards, > > Fokko > > > > On 2026/07/22 21:43:03 Kevin Liu wrote: > > > I found a potential issue with the license and notice files that might > > be a > > > release blocker. > > > > > > Many of the jars lack license files, notice files, or both. > > > Out of 59 staged JARs: > > > - 40 lack both files > > > - 2 lack LICENSE > > > - 10 lack NOTICE > > > The remaining 7 contain both, but their contents might be incomplete or > > > outdated. > > > > > > Here's the script I used to verify and the result from the audit: > > > > > > https://gist.github.com/kevinjqliu/2f899c085abb2716b13c220d4f52a677?permalink_comment_id=6271325#gistcomment-6271325 > > > > > > I also found that the NOTICE file's copyright year is out of date ( > > > > > > https://github.com/apache/parquet-java/blob/83c2c80d49c57b1ae180df45229ee01dbceddc1e/NOTICE#L3 > > > ) > > > > > > This is my first time verifying a parquet-java release, so please let > me > > > know if there's anything I'm missing. > > > > > > Best, > > > Kevin Liu > > > > > > On Wed, Jul 22, 2026 at 1:30 PM Steve Loughran <[email protected]> > > wrote: > > > > > > > +1 non-binding. > > > > > > > > I only audited the tarball and the maven artifacts, not the code; > > leaving > > > > that to others. I was answering the question "did the release-manager > > place > > > > anything malicious in the nexus artifacts", which is something you > > should > > > > be doing for all releases these days. > > > > > > > > Verified the RC1 source tarball and staged Maven artifacts: > > > > * Source tarball SHA-512 OK; GPG signature good (Fokko Driesprong, > key > > 2F37 > > > > FE05 5E54 7EDD ACD9 A633 B13C 6981 6C5E 13EF). Signed Fokko's key > BTW, > > as > > > > nobody else has. > > > > * Extracted source matches tag apache-parquet-1.18.0-rc1 exactly > > > > * All 15 staged modules (104 artifacts): SHA-1/MD5 checksums match > and > > > > every GPG signature is good > > > > * Rebuilt from the tag with JDK 11 (mvn clean package): > > parquet-encoding > > > > and parquet-jackson jars are byte-identical to the release; the rest > > differ > > > > only in bytecode traceable to a javac string-concat idiom > > (String.valueOf + > > > > makeConcatWithConstants). > > > > * No structural differences (no added/removed classes, methods, or > > fields) > > > > and no new suspicious API calls in any jar. > > > > > > > > For anyone who wants to do that jar comparison, Auditor: > > > > https://github.com/steveloughran/auditor > > > > This disassembles the binary and looks at the bytecodes, so doesn't > > require > > > > a byte for byte match. Give it to claude to validate (as done here) > and > > > > it'll analyse why there there's a mismatch. > > > > > > > > steve > > > > > > > > > > > > On Mon, 20 Jul 2026 at 15:30, Fokko Driesprong <[email protected] > > > > > > wrote: > > > > > > > > > Hi everyone, > > > > > > > > > > I propose the following RC to be released as official Apache > Parquet > > > > > 1.18.0 release. > > > > > > > > > > - The commit id is [5e2ad0969c7f411613671c5ef054d399995de5f9]( > > > > > > > > > > > > https://github.com/apache/parquet-java/tree/5e2ad0969c7f411613671c5ef054d399995de5f9 > > > > > ) > > > > > - This corresponds to the tag: [apache-parquet-1.18.0-rc1]( > > > > > > > > > > > > https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.18.0-rc1 > > > > > ) > > > > > > > > > > The release tarball, signature, and checksums are here: > > > > > > > > > > - > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/parquet/apache-parquet-1.18.0-rc1/ > > > > > > > > > > You can find the KEYS file here: > > > > > > > > > > - https://downloads.apache.org/parquet/KEYS > > > > > > > > > > You can find the changelog here: > > > > > > > > > > - > > > > > > > > > > > > https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.18.0-rc1 > > > > > > > > > > Binary artifacts are staged in Nexus here: > > > > > > > > > > - > > > > > > https://repository.apache.org/content/groups/staging/org/apache/parquet/ > > > > > > > > > > Please download, verify, and test. > > > > > > > > > > Please vote in the next 72 hours. > > > > > > > > > > [ ] +1 Release this as Apache Parquet 1.18.0 > > > > > [ ] +0 > > > > > [ ] -1 Do not release this because... > > > > > > > > > > Kind regards, > > > > > Fokko Driesprong > > > > > > > > > >
