ErikBPF opened a new pull request, #3796: URL: https://github.com/apache/parquet-java/pull/3796
### Rationale for this change Closes #3795. Duplicate primitive root names must not let name-path lookup substitute a different physical leaf's type, pages, or indexes. The consumer reproduction and standalone fixture source are in apache/datafusion-comet#5964: https://github.com/apache/datafusion-comet/issues/5964. ### What changes are included in this PR? - Resolve footer metadata using schema leaf ordinal; validate row-group column count and available physical path/type before path construction and crypto registration. - Preserve the first chunk and matching index for duplicate primitive root paths in ordinary and filtered row-group reads. - Retain empty-schema projections and lazy column-key metadata decryption. - Add 13 JUnit 5/AssertJ regression tests with generated fixtures, distinct page layouts, partial indexed reads, malformed controls, and encryption controls. This targets master at `7a9665a392551143548477f733c9321c4775b32e`. The initial investigation used parquet-java 1.16.0; this port preserves master's writer-version handling and uses its Java 17/JUnit 5 toolchain. No release pin or Java 8 compatibility claim is made. ### Are these changes tested? On JDK 17, Maven 3.9.11, and required Thrift 0.24.0: ```sh export MAVEN_OPTS='-Xmx2g -XX:ActiveProcessorCount=2' export JAVA_TOOL_OPTIONS='-Xmx2g -XX:ActiveProcessorCount=2' timeout 600 ./mvnw -B -ntp -pl parquet-hadoop -am spotless:check package \ -Dtest=TestDuplicateRootIdentity,TestParquetMetadataConverter,TestColumnIndexFiltering,TestEncryptionOptions \ -Dsurefire.failIfNoSpecifiedTests=false ``` The equivalent Maven 3.9.11 invocation passed in 67 seconds: **112 tests, zero failures/errors/skips** (metadata 74, index filtering 24, duplicate identity 13, encryption options 1). Spotless and normal lifecycle checks passed. The selected module's prerequisite modules were built; their full test suites and the full repository reactor were not run. Earlier exact-release consumer validation passed Spark's duplicate suite (8 tests) and V1/V2 filter suites (101 tests), but that result is not validation of this current-master port or a published downstream dependency. ### Are there any user-facing changes? Duplicate primitive root selection retains the first physical chunk and corresponding index rather than overwriting it with a later namesake. Invalid footer count/path/type combinations fail with `ParquetDecodingException`. No general nested/encrypted duplicate support is claimed. A downstream Spark fix still needs coordinated descriptor/filter handling and a normally published Parquet artifact; this PR alone does not claim to fix every consumer path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
