pingzh opened a new pull request, #5604: URL: https://github.com/apache/datafusion-comet/pull/5604
## Which issue does this PR close? Closes #5530. ## Rationale for this change The native Celeborn shuffle path reflects on private Celeborn members. Its ordinary unit tests use hand-written stand-ins, so they cannot detect when a released Celeborn client changes one of those fields or method signatures. ## What changes are included in this PR? - Add an opt-in `celeborn-reflection-compatibility` Maven profile with a test-scoped shaded Celeborn client dependency and an isolated test source directory. - Add a reflection-only suite that verifies every Celeborn field and method used by the partition pusher and callback tracker, including the version-specific `mapperEnd`, CRC, and crypto differences. - Run the suite in a dedicated Linux CI matrix against Celeborn 0.6.0 and 0.7.0. - Keep the compatibility suite out of normal builds and the generic suite matrix. ## How are these changes tested? - Clean compatibility-suite runs against Celeborn 0.6.0 and 0.7.0 (4 tests each). - Clean profile-free test compilation with Spark 3.5 and Scala 2.12, confirming the isolated suite is not compiled normally. - Spotless formatting check. - `python3 dev/ci/check-suites.py`. - Workflow YAML parsing and `git diff --check`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
