andygrove opened a new pull request, #7560: URL: https://github.com/apache/datafusion-comet/pull/7560
## Which issue does this PR close? Part of #4329. ## Rationale for this change #7521 removes the `spark-3.4` Maven profile and stops running Spark 3.4 in CI, but leaves the Spark 3.4 shim source sets behind. Nothing builds them now, so they are dead code that has to be kept in step with the other shims. This is the follow-up that deletes them. **This PR is stacked on #7521.** It contains that PR's three commits followed by one new commit, so until #7521 merges the diff shows both. Review only the last commit, `chore: remove the Spark 3.4 shim source sets`. Once #7521 merges I will rebase onto `main` and the diff will shrink to that commit. ## What changes are included in this PR? - Delete `spark/src/main/spark-3.4/` and `spark/src/test/spark-3.4/`. Every file in them has a counterpart in `spark-3.5/` or is a 3.5-only addition there, so the `spark-3.5`, `spark-3.x` and `spark-4.x` source sets are unchanged. - Remove the `spark-3.4` path entries from `dev/ci/compute-changes.py`. They only excluded edits to the deleted directory from triggering other jobs. - Update comments that described 3.4 (`ShimCometMapInBatch`, `GenerateDocs`). - Update the contributor and user docs that described Spark 3.4 behavior or listed `spark-3.4` as a shim directory, and redraw `shim_pattern.svg` with the 3.5 and 4.x editions. Deliberately not touched: - The native `get_json_object_spark34` function. The Spark 3.4 shim was its only caller, but it is a named behavior flag that #6891 already tracks renaming, so it is better handled there. - `dev/diffs/3.4.3.diff` and `dev/diffs/iceberg/1.8.1.diff`, as in #7521. - The `"3.4.3"` cases in the version-parsing unit tests and the `bit_count_types.sql` interpreted-mode workaround. They are harmless and can go in a later cleanup. - The expression-audit pages and changelogs, which record what was true for each Spark version at the time. ## How are these changes tested? - `python3 dev/ci/check-ci-config.py`, `check-versioned-source-parity.py` and `check-suites.py` pass. - `prettier --check` passes on the changed Markdown, and `shim_pattern.svg` parses as XML. - Not run: a Maven build. No remaining profile compiles the deleted directories, and the only Scala changes are comments. CI on this PR exercises the remaining profiles. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
