kerwin-zk opened a new pull request, #7721: URL: https://github.com/apache/paimon/pull/7721
### Purpose Follow-up to #7648 (Spark 4.1 module). After the reverse-shim layout landed, three of the files copied into `paimon-spark-4.0/src/main` only differed across versions because of `case class .copy(...)` calls on Spark types whose arity changed between 4.0.2 and 4.1.1: - `DataSourceV2Relation` gained `Option[TimeTravelSpec]` (8 → 9 fields) — `relation.copy(table = ...)` compiled against 4.1.1 emits `copy\$default\$9`, which crashes on 4.0 with `NoSuchMethodError`. - `TableSpec` gained `Seq[Constraint]` (8 → 9 fields) — same problem for `spec.copy(location = ...)` and `spec.copy(properties = ...)`. Per-version scalac is the only thing that knows the right `copy\$default\$N` to emit, so we route those three calls through new `SparkShim` factories (one per call site). The implementations live in `Spark3Shim` / `Spark4Shim` (plus the 4.0 override), and the cross-version source files no longer need to be physically duplicated. ### Linked issue Follow-up of #7648. ### Tests - `paimon-spark-4.1` full UT: 36 suites / 728 succeeded / 0 failed - `paimon-spark-4.0` full UT: 36 suites / 727 succeeded / 1 canceled / 0 failed - Verified on top of latest master (post-#7648 merge). ### API and Format No new public API. Three internal helper methods added to `org.apache.spark.sql.paimon.shims.SparkShim`: - `copyDataSourceV2Relation(relation, newTable)` - `copyTableSpecLocation(spec, location)` - `copyTableSpecProperties(spec, properties)` ### Documentation No user-facing changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
