nsivabalan commented on PR #19082:
URL: https://github.com/apache/hudi/pull/19082#issuecomment-4859625265
⚠️ **Coverage gap: version-specific test classes in `hudi-sparkX.Y.x`
submodules run on zero Spark versions after this PR.**
The version-specific test modules (`hudi-spark3.3.x`, `hudi-spark3.4.x`,
`hudi-spark4.0.x`, `hudi-spark4.1.x`) exist precisely because they exercise
Spark-version-pinned integration points. On master today, each of these classes
runs on its own Spark version as part of the full per-version datasource job.
After this PR, those per-version jobs are trimmed to `spark3.5` (Java 11) and
`spark4.2` (Java 17) only, and the new `test-spark-core-tests` job runs
surefire `groups=core` in those same modules — but **none of the
version-specific tests carry `@Tag("core")`**, so surefire finds zero matches
and they stop running everywhere.
Concretely, the test classes that run on **zero** Spark versions after this
PR:
-
`hudi-spark-datasource/hudi-spark3.3.x/src/test/scala/org/apache/hudi/TestHoodieStreamingSinkConstants.scala`
-
`hudi-spark-datasource/hudi-spark3.4.x/src/test/scala/org/apache/hudi/TestHoodieStreamingSinkConstants.scala`
-
`hudi-spark-datasource/hudi-spark4.0.x/src/test/scala/org/apache/hudi/TestHoodieStreamingSinkConstants.scala`
-
`hudi-spark-datasource/hudi-spark4.1.x/src/test/scala/org/apache/hudi/TestHoodieStreamingSinkConstants.scala`
-
`hudi-spark-datasource/hudi-spark4.0.x/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/TestSpark40HoodieParquetReadSupport.scala`
-
`hudi-spark-datasource/hudi-spark4.0.x/src/test/java/org/apache/hudi/io/storage/row/TestHoodieRowParquetWriteSupportVariant.java`
-
`hudi-spark-datasource/hudi-spark4.1.x/src/test/scala/org/apache/hudi/TestSpark4_1AvroLogicalTypeBytes.scala`
`TestHoodieStreamingSinkConstants` is the clearest example — its docstring
says *"Validates that `HoodieStreamingSink.QUERY_ID_KEY` matches
`StreamExecution.QUERY_ID_KEY` from this Spark version."* The whole point of
the test is that it's Spark-version-pinned; if Spark quietly renames or
renumbers the private field on 3.3, 3.4, 4.0, or 4.1, CI no longer catches it.
**Suggested fix:** add `@Tag("core")` at the class level to each of the
classes above so they continue to run on their own Spark version via
`test-spark-core-tests`. It's a one-line change per file:
```scala
@Tag("core")
class TestHoodieStreamingSinkConstants { ... }
```
Cheap to keep, high signal on the exact drift they're written to catch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
