andygrove commented on code in PR #7521:
URL: https://github.com/apache/datafusion-comet/pull/7521#discussion_r4486522866
##########
dev/ci/linux-test-profiles.py:
##########
@@ -46,7 +46,6 @@
# `run-all-spark-profiles` label). The goal of the list is coverage of every
# Java, Scala and Spark version without testing every combination.
PROFILES = [
- {"name": "Spark 3.4, JDK 17, Scala 2.12", "java_version": "17",
"maven_opts": "-Pspark-3.4 -Pscala-2.12", "tier": "nightly"},
{"name": "Spark 3.5, JDK 17, Scala 2.13", "java_version": "17",
"maven_opts": "-Pspark-3.5 -Pscala-2.13", "tier": "nightly"},
Review Comment:
Added a `Spark 3.5, JDK 17, Scala 2.12` nightly row next to the 2.13 one, so
Scala 2.12 keeps running the Comet suites. Pushed in 38192970de.
##########
dev/regenerate-golden-files.sh:
##########
@@ -129,7 +130,7 @@ main() {
if [ -n "$target_version" ]; then
if [[ ! "$target_version" =~ ^(3\.4|3\.5|4\.0|4\.1|4\.2)$ ]]; then
Review Comment:
Dropped `3\.4|` from the pattern, so `--spark-version 3.4` now hits the
invalid-version error. Pushed in 38192970de.
##########
docs/source/user-guide/latest/installation.md:
##########
@@ -47,15 +47,8 @@ Other versions may work well enough for development and
evaluation purposes.
Comet requires JDK 17 or later. JDK 11 is no longer supported as of the 1.1.0
release.
-```{warning}
-Spark 3.4 support is deprecated as of the 1.0.0 release and will be removed in
a future release.
-Apache Spark's own SQL test suite no longer runs against Spark 3.4
automatically; it runs only on
-demand. We recommend moving to Spark 3.5 or later.
-```
-
| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark SQL
Tests in CI |
| ------------- | ------------ | ------------- | ----------------- |
--------------------- |
-| 3.4.3 | 17 | 2.12/2.13 | Nightly | On demand
|
| 3.5.9 | 17 | 2.12/2.13 | Nightly | Nightly
|
Review Comment:
Added an `Upgrading to Comet 1.3.0` section to `migration-guide.md` saying
Spark 3.4 is no longer built or published and that 3.4 users should stay on the
last release with `comet-spark-spark3.4_*`. Pushed in 38192970de.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
