rdtr opened a new pull request, #13591: URL: https://github.com/apache/gluten/pull/13591
## What changes are proposed in this pull request? Bump the `spark-4.2` profile from Spark 4.2.0 to 4.2.1 (tag v4.2.1). Same pattern as the 4.0.2 bump (#12180). - `pom.xml` (`spark-4.2` profile) and `tools/gluten-it/pom.xml`: `spark.version` 4.2.1. Spark's own dependency versions (Scala 2.13.18, Jackson 2.21.2, Arrow 19.0.0, Hadoop 3.5.0, ...) are the same in v4.2.0 and v4.2.1, so no other pins move. - `.github/workflows/util/install-spark-resources.sh`: download the 4.2.1 binary and source tarballs. - `velox_backend_x86.yml` step name and one docs example. - `gluten-ut/spark42` `VeloxTestSettings`: exclude the tests that 4.2.1 added or changed and that fail under Gluten today. ### Excluded tests (Spark 4.2.1) - **Cast string to decimal with huge exponents** (SPARK-60118, SPARK-60119, and the extended "Fast fail for cast string type to decimal type" tests, in `GlutenCastWithAnsiOffSuite`, `GlutenCastWithAnsiOnSuite`, `GlutenTryCastSuite`): inputs such as `1e2147483647` crash the JVM with SIGSEGV in `DecimalUtil::parseStringToDecimalComponents`. This is a Velox bug, and it kills the whole test JVM. Needs a Velox fix; I will follow up. - **Array set operations** (SPARK-59602, 2 tests): Spark now normalizes -0.0 and NaN in the result; Velox does not. - **overlay, lpad/rpad** (SPARK-58708, SPARK-58713, 5 tests): Spark fixed length and position arithmetic; Velox differs. - **Storage-partitioned join** (`GlutenKeyGroupedPartitioningSuite`, 23 tests) and SPARK-59141 (`GlutenDataFrameSetOperationsSuite`): the tests check vanilla plans (e.g. `VeloxColumnarToRow` is not `SafeForKWayMerge`). Not touched: `SPARK-35711` and `SPARK-35719` in the cast suites fail the same way on 4.2.0 on my machine, so they are not part of this bump. ### Conflicts #13519 and #13520 `git mv` spark42 to spark43 and copy it back. Whichever lands second rebases; this diff is small. ## How was this patch tested? `mvn test -pl gluten-ut/spark42 -Pspark-4.2 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Pspark-ut -Pdelta` with `-Dspark.test.home` at the v4.2.1 sources, on `org.apache.spark.sql.catalyst.expressions`, `GlutenStringFunctionsSuite`, `GlutenDataFrameSetOperationsSuite` and `GlutenKeyGroupedPartitioningSuite`: 1687 passed, only SPARK-35711/35719 fail (same on 4.2.0). The remaining gluten-ut groups are left to CI. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Sonnet 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
