manoj-ragupathy opened a new pull request, #13022: URL: https://github.com/apache/gluten/pull/13022
## What changes are proposed in this pull request? > **Stacked on #13020 → #13021.** Targets `main`, so the diff currently also shows the earlier commits in the stack. **Only the top 6 commits belong to this PR.** Draft until its parents merge. > ### ⚠️ Please rebase-merge this PR, do not squash > The bulk of this diff is a **pure rename** of `gluten-ut/spark41` → `gluten-ut/spark42`, kept as its own commit so `git log --follow` continues to trace each file into its pre-4.2 history. Squashing collapses the rename together with the content edits and destroys rename detection. This follows the commit-history guidance in #11352. I verified `git log --follow` resolves through the move commit into the original 4.1 history. Third step of Spark 4.2.x support (#12569): add the `gluten-ut/spark42` unit-test module and turn on the corresponding CI jobs. - Copy `gluten-ut/spark41` → `gluten-ut/spark42` as a standalone rename commit (~1850 files, almost entirely mechanical). - Adapt the suites that Spark 4.2 broke, and refresh `VeloxTestSettings` / `ClickHouseTestSettings` for tests added, removed or renamed between 4.1.1 and 4.2.0. - Add the Spark 4.2 UT jobs to `velox_backend_x86.yml`, matching the existing 4.1 lanes. Each generated wrapper was checked to extend the correct Gluten trait — #11800 showed that getting this wrong makes suites silently run on vanilla Spark without the plugin, so they pass while testing nothing. Part of #12569. ## How was this patch tested? This PR *is* the test module. Full reactor build: ```bash ./build/mvn -Pspark-4.2 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Pspark-ut clean install -DskipTests ``` `BUILD SUCCESS` — every suite compiles against Spark 4.2.0. **Honest caveat:** I could not execute the suites locally (no `libgluten.so` in my environment), so the actual pass/fail signal has to come from CI. And the 4.2 UT lanes cannot run until the CI image is rebuilt with the Spark 4.2 resources that #13021 adds to the Dockerfile — `docker_image.yml` only rebuilds on a Sunday cron, so **a committer needs to `workflow_dispatch` it first** (also noted on #12569). I expect the first real run to surface failures needing `VeloxTestSettings` adjustments, and I will iterate on those. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: GitHub Copilot CLI (Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
