LuciferYang opened a new pull request, #12886:
URL: https://github.com/apache/gluten/pull/12886
## What changes are proposed in this pull request?
`VeloxTestSettings` excludes six cases from `GlutenTryCastSuite` on Spark
3.4, 3.5, 4.0 and 4.1. All six pass once the exclusions are removed, so this
deletes them. Unlike the neighbouring entries in that block, none of the six
carries a comment saying what fails.
```scala
.exclude("ANSI mode: Throw exception on casting out-of-range value to byte
type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to short
type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to int
type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to long
type")
.exclude("cast from invalid string to numeric should throw
NumberFormatException")
.exclude("SPARK-26218: Fix the corner case of codegen when casting float to
Integer")
```
Why they were likely added: on Spark 3.3 these cases live in
`AnsiCastSuiteBase`, and `TryCastSuite` builds them through `TryCast`, a
3.3-only expression class. Gluten's expression mapping is keyed by class,
`Sig(expClass: Class[_], name: String)`, and there is no `Sig[TryCast]`, so a
plan containing `TryCast` cannot be offloaded on 3.3 and falls back to vanilla
Spark. A 3.3 run logs `Skipping evaluation - Nonempty inputRow cannot be
converted to DataFrame` ten times for this suite, and Gluten's own try_cast
validation is gated the same way with `testWithMinSparkVersion("try_cast",
"3.4")` in `ScalarFunctionsValidateSuite`. Spark 3.4 replaced `TryCast` with
`Cast(child, dataType, timeZoneId, EvalMode.TRY)`, which Gluten does offload
(`Spark34Shims.withTryEvalMode`, `UnaryExpressionTransformer`,
`VeloxSparkPlanExecApi`), so the cases went from never reaching Velox to
genuinely exercising it. Whatever failed at that point passes now.
Fixes #12863's one actionable finding. That issue asks whether removing
`gluten-ut/spark33` under #12807 would drop coverage that only runs on 3.3. The
inventory behind it produced 17 candidate cases; these six are the only ones
that pass once enabled. The rest either fail in a way consistent with the
reasons written beside them, or are ClickHouse-only and need a CH build to
check. So the answer to #12863 is that nothing is lost by removing the module,
and the one thing worth doing is re-enabling these six.
## How was this patch tested?
Draft on purpose: CI on all four versions is the check that matters here,
not my laptop.
Removing the six from `gluten-ut/spark34` and running the suite against
Spark 3.4 locally takes it from 72 to 78 cases with no new failures, and all
six report as passed. The delta is what makes that readable: before the change
these cases do not run at all, so they cannot appear in a baseline failure set,
and afterwards they run and pass.
The same local run also fails `Gluten - data type casting` and `Gluten -
cast string to timestamp`, and a companion run of `GlutenParquetIOSuite` fails
five more cases. All of those are pre-existing, timezone or parquet related,
and green in CI, so this machine deviates from CI and a local result is not
authoritative on its own. Hence the draft.
`test-compile` passes on `-Pspark-3.4`, `-Pspark-3.5 -Pscala-2.13`,
`-Pspark-4.0 -Pscala-2.13` and `-Pspark-4.1 -Pscala-2.13`, each with
`-Pbackends-velox -Pspark-ut -Pjava-17`, and `spotless:check` is clean.
If CI red-flags any of the six on a particular version, the intent is to put
the exclusion back for that version with the failure written next to it, rather
than to drop the whole change.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude claude-opus-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
