rdtr opened a new pull request, #13591:
URL: https://github.com/apache/gluten/pull/13591

   ## What changes are proposed in this pull request?
   
   Bump the `spark-4.2` profile from Spark 4.2.0 to 4.2.1 (tag v4.2.1). Same 
pattern as the 4.0.2 bump (#12180).
   
   - `pom.xml` (`spark-4.2` profile) and `tools/gluten-it/pom.xml`: 
`spark.version` 4.2.1. Spark's own dependency versions (Scala 2.13.18, Jackson 
2.21.2, Arrow 19.0.0, Hadoop 3.5.0, ...) are the same in v4.2.0 and v4.2.1, so 
no other pins move.
   - `.github/workflows/util/install-spark-resources.sh`: download the 4.2.1 
binary and source tarballs.
   - `velox_backend_x86.yml` step name and one docs example.
   - `gluten-ut/spark42` `VeloxTestSettings`: exclude the tests that 4.2.1 
added or changed and that fail under Gluten today.
   
   ### Excluded tests (Spark 4.2.1)
   
   - **Cast string to decimal with huge exponents** (SPARK-60118, SPARK-60119, 
and the extended "Fast fail for cast string type to decimal type" tests, in 
`GlutenCastWithAnsiOffSuite`, `GlutenCastWithAnsiOnSuite`, 
`GlutenTryCastSuite`): inputs such as `1e2147483647` crash the JVM with SIGSEGV 
in `DecimalUtil::parseStringToDecimalComponents`. This is a Velox bug, and it 
kills the whole test JVM. Needs a Velox fix; I will follow up.
   - **Array set operations** (SPARK-59602, 2 tests): Spark now normalizes -0.0 
and NaN in the result; Velox does not.
   - **overlay, lpad/rpad** (SPARK-58708, SPARK-58713, 5 tests): Spark fixed 
length and position arithmetic; Velox differs.
   - **Storage-partitioned join** (`GlutenKeyGroupedPartitioningSuite`, 23 
tests) and SPARK-59141 (`GlutenDataFrameSetOperationsSuite`): the tests check 
vanilla plans (e.g. `VeloxColumnarToRow` is not `SafeForKWayMerge`).
   
   Not touched: `SPARK-35711` and `SPARK-35719` in the cast suites fail the 
same way on 4.2.0 on my machine, so they are not part of this bump.
   
   ### Conflicts
   
   #13519 and #13520 `git mv` spark42 to spark43 and copy it back. Whichever 
lands second rebases; this diff is small.
   
   ## How was this patch tested?
   
   `mvn test -pl gluten-ut/spark42 -Pspark-4.2 -Pscala-2.13 -Pjava-17 
-Pbackends-velox -Pspark-ut -Pdelta` with `-Dspark.test.home` at the v4.2.1 
sources, on `org.apache.spark.sql.catalyst.expressions`, 
`GlutenStringFunctionsSuite`, `GlutenDataFrameSetOperationsSuite` and 
`GlutenKeyGroupedPartitioningSuite`: 1687 passed, only SPARK-35711/35719 fail 
(same on 4.2.0). The remaining gluten-ut groups are left to CI.
   
   ## Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Code (Claude Sonnet 5.5)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to