This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch branch-4.2 in repository https://gitbox.apache.org/repos/asf/doris.git
commit e903b027cac4cf734fd8cd6927d4a85661a00c99 Author: Gabriel <[email protected]> AuthorDate: Sat Oct 10 15:24:01 2026 +0800 [fix](hudi) Align preloaded Parquet format classes on branch-4.1 (#68856) ### What problem does this PR solve? Hudi JNI scans on branch-4.1 fail with `NoSuchMethodError: org.apache.parquet.format.Statistics.isSetNan_count()` after the Parquet 1.18.0 upgrade (#68557). The BE preload assembly unpacks both the explicitly pinned `parquet-format:2.12.0` and `parquet-hadoop-bundle:1.18.0`. The former supplies an older `Statistics` class, while the latter supplies a reader that calls a method absent from that class. The scanner inherits these incompatible classes from its parent classloader. Remove the redundant standalone format dependency so the format classes and readers come from the same Parquet bundle. Keep the existing Parquet version and classloader behavior. ### Release note Fix Hudi JNI scan failures caused by incompatible Parquet classes in the BE preload jar. ### Check List (For Author) - Test - [x] Manual test (details below) - Behavior changed: - [x] Yes. Hudi JNI scans can convert Parquet statistics without the missing-method error. - Does this need documentation? - [x] No. ### Validation - Assembled the unmodified preload module and reproduced the missing-method error by converting INT64 statistics. - Cleanly reassembled the modified module with `mvn -pl be-java-extensions/preload-extensions clean assembly:single@make-assembly-deps` from `fe/`. - Ran the same statistics conversion against the rebuilt jar, with Hadoop Common and JTS Core runtime dependencies on the classpath; it passed. Both `Statistics` and `ParquetMetadataConverter` loaded from the rebuilt preload jar. - Compared all 1,795 `org/apache/parquet/**/*.class` entries from the bundle against the rebuilt assembly; all matched byte for byte. - Passed `mvn -pl be-java-extensions/preload-extensions -am checkstyle:check` and checked the reactor dependency tree. The statistics conversion probe uses: ```java Statistics stats = new Statistics(); stats.setNull_count(0); ParquetMetadataConverter.fromParquetStatistics( "parquet-mr version 1.18.0", stats, PrimitiveTypeName.INT64); ``` Full Hudi integration tests were not run locally. Existing Hudi catalog, incremental, time-travel, snapshot, and schema-evolution regression suites cover the affected JNI path and should be rerun in CI. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label --- fe/be-java-extensions/preload-extensions/pom.xml | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/fe/be-java-extensions/preload-extensions/pom.xml b/fe/be-java-extensions/preload-extensions/pom.xml index 772a98dd653..673069eb2d8 100644 --- a/fe/be-java-extensions/preload-extensions/pom.xml +++ b/fe/be-java-extensions/preload-extensions/pom.xml @@ -36,11 +36,7 @@ under the License. </properties> <dependencies> - <dependency> - <groupId>org.apache.parquet</groupId> - <artifactId>parquet-format</artifactId> - <version>2.12.0</version> - </dependency> + <!-- Keep Parquet format classes and readers in the same bundle to avoid binary incompatibility. --> <dependency> <groupId>org.apache.parquet</groupId> <artifactId>parquet-hadoop-bundle</artifactId> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
