LuciferYang opened a new pull request, #9794:
URL: https://github.com/apache/paimon/pull/9794
### Purpose
Closes #9793.
`paimon-spark-4.0` takes the shared test bases from `paimon-spark-ut`'s
test-jar, which is compiled against the `spark4` profile baseline (4.1.2). Test
bytecode built against 4.1.2 can name Spark symbols that a 4.0.3 runtime cannot
resolve, and `StreamTestCheckAnswerWithTimeoutStub` covered that by hand: eight
empty classes whose compiled names match `StreamTest` inner classes that exist
only from Spark 4.1 on. 15 classfiles in the baseline test-jar reference those
names, none of them from anything a Paimon test calls; scalac emits them into
the constant pool through `StreamTest`'s mix-in forwarders.
`paimon-spark-ut-4.0` recompiles the same sources against 4.0.3 and produces
a test-jar only. It owns no sources: `build-helper` adds
`../paimon-spark-ut/src/test/{scala,java}`, so there is still exactly one copy
of every test to maintain. A class compiled against 4.0.3 cannot reference a
symbol 4.0.3 does not have, so the stub is deleted rather than extended the
next time the baseline moves.
`skipTests` is a module property rather than scalatest plugin configuration.
Surefire inherits the root pom's `default-test` and `integration-tests`
executions, and both plugins bind their `skipTests` parameter to
`${skipTests}`, so the property covers both. Without it, `mvn install` over the
reactor without `-DskipTests` runs the shared Java tests a second time in this
module and fails on `TestLoggerExtension` with `NoClassDefFoundError:
org/slf4j/spi/LoggingEventBuilder`, since the module carries no SLF4J 2.x pin
of its own. `maven.test.skip` is not usable here: it would skip test
compilation and leave the test-jar empty.
`paimon-spark-4.1` is untouched. While the baseline is 4.1.x, the baseline
build already matches its runtime, so it keeps consuming `paimon-spark-ut`
directly.
### Tests
On JDK 17:
- `mvn -Pflink1,spark4 clean install -DskipTests` over
`paimon-spark-ut-4.0`, `paimon-spark-4.0` and `paimon-spark-4.1` with `-am`:
success.
- `paimon-spark-4.0` full suite with the stub gone: 44 suites, 1217 tests
succeeded, 22 version-gated cancellations, no failures and no aborted suites.
- The 4.0.3-compiled test-jar and the baseline one carry the same 584
classes: the two name sets were compared and both differences are empty. The
eight stub symbols appear in 15 baseline classfiles and in none of the
4.0-compiled ones.
- `mvn -pl paimon-spark-ut-4.0 -Pflink1,spark4 install` without
`-DskipTests`: success, three `Tests are skipped.`, 584 test classes still
compiled, test-jar still 629 entries. Before `skipTests` was added, the same
command failed with the `NoClassDefFoundError` above and exit 1, which is how
that defect was found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
