JingsongLi opened a new pull request, #11127: URL: https://github.com/apache/paimon/pull/11127
### Purpose Add an experimental `paimon-spark-4.2_2.13` connector through an independent `spark42` Maven profile. It compiles the shared connector modules against Spark 4.2.0 / Scala 2.13.18 while keeping the existing `spark4` baseline and Spark 4.0/4.1 build paths. This is a smaller alternative to #9265: it does not raise the shared baseline for all Spark 4.x artifacts. Compatibility changes cover output resolution, catalog commands, function parsing/registration, CHAR/VARCHAR construction, staged writes, scan matching, and Spark 4.2 BinaryView spatial getters. A small 4.2-only rule aligns fresh UPDATE/MERGE commands and delegates to Spark's native row-level rewriters. The profile has its own CI lane and a documented source-build command. Connector artifacts from different profiles must be built separately with `clean`, since the shared Maven artifact names are unchanged. Snapshot/release publication and semantic-view execution are outside this PR. ### Tests - `mvn -pl paimon-spark/paimon-spark-4.2 -am -Pspark42 -DfailIfNoTests=false -Dtest=none -DwildcardSuites=org.apache.paimon.spark.sql.Spark42CompatibilityTest package` after cleaning the Spark modules: passed without `fast-build`. - 15 focused compatibility tests: CTAS, static/dynamic partition overwrite, views, partition description, scalar SQL functions, V1/V2 UPDATE/DELETE/MERGE on primary-key, ordinary append, row-tracking, deletion-vector and Data Evolution tables, subquery UPDATE, insert-only MERGE, VARIANT and spatial scalar/array reads. - Repeated the same 15 tests with the shaded connector JAR first on the runtime classpath: passed. - `python3 -m unittest discover -s tools/ci -p 'test_*.py'`: 30 passed. - Root Apache RAT check and `git diff --check`: passed. This initial source-build PR has focused compatibility coverage; the complete Spark engine test suite has not been run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
