rdtr opened a new pull request, #13589:
URL: https://github.com/apache/gluten/pull/13589

   Branch: rdtr:spark43-golden-files, base apache/gluten main (stacked on 
#13520)
   
   ## What changes are proposed in this pull request?
   
   Depends on #13520. Part of #13517. Only the last commit belongs to this PR: 
https://github.com/rdtr/incubator-gluten/commit/aa25933200397131d27f39032673e107e2e3548b
   
   #13520 removes 21 files from `SUPPORTED_SQL_QUERY_LIST` in 
`gluten-ut/spark43` without a comment (cast, decimalArithmeticOperations, time, 
like-any / like-all / ilike-any / ilike-all, nonansi/cast, postgreSQL/date and 
others). Six more are left out with `TODO: fix on Spark-4.3` (keywords, 
postgreSQL/float8, nonansi/keywords, identifier-clause-legacy, datetime-legacy, 
postgreSQL/window_part3); this PR doesn't touch those.
   
   All 21 failed on the first query whose output Spark 4.3 changed. The cause 
is that Gluten keeps its own golden files under `backends-velox/sql-tests`, and 
in #13520 they are still the Spark 4.2 copies. For example, `time.sql` gets new 
TIME(p) cases, `cast.sql` gets new TIME and nanos cases, and the `like-any` 
family changes an error class.
   
   This replaces the inputs and results of the 21 files with the Spark 4.3.0 
ones and puts Gluten's own edits back on top:
   
   - `having-and-order-by-recursive-type-name-resolution.sql`: the NullType 
query (SPARK-57186) stays removed, as in the 4.2 copy.
   - `type-coercion-edge-cases.sql`: Gluten's changed result from the 4.2 copy 
(`SELECT col2.field FROM t1` returns a `double` result in Velox where Spark 
throws `CAST_INVALID_INPUT`) is applied on top of the 4.3 file.
   - `time.sql`: the numeric constructor section (`time_from_seconds` onward) 
stays cut, as in the 4.2 copy, which also drops the SPARK-57585 mixed-precision 
cases that follow it. The two new SPARK-58296 queries 
`typeof(to_time('00:12:00', null))` are also removed. They fall back, and 
Spark's `TypeOf` codegen emits the unquoted type name (`time`) and fails to 
compile with `A method named "time" is not declared`. Gluten's SQL query suite 
excludes `ConstantFolding`, so the query is never folded as it is in Spark.
   
   Three files stay out of the list, with a comment naming the cause:
   
   - `cast.sql` and `nonansi/cast.sql`: query #70, `cast('2022-01-01 
00:00:00.123456789' as timestamp_ntz(9))`, falls back and fails in codegen the 
same way. It needs the nanosecond timestamp types to be handled, which is 
outside this PR. Their goldens are already the 4.3 ones, so enabling them later 
is a one-line change.
   - `nonansi/try_aggregates.sql`: `SELECT try_sum(col) FROM VALUES (5.0), 
(10.0), (15.0) AS tab(col)` fails in Velox with `Reason: (ROW vs. FLAT)` from 
`SimpleNumericAggregate.h:50` (`doExtractValues`) in `PartialAggregation`. The 
file is enabled on Spark 4.2, so I'll look at it separately.
   
   None of the 18 needed a Velox change.
   
   ## How was this patch tested?
   
   `GlutenSQLQueryTestSuite` on Spark 4.3.0 (`-Pspark-4.3 -Pspark-ut`, 
`spark.test.home` at the v4.3.0 source): 229 succeeded, 0 failed. Before the 
golden update, all 21 files failed (211 succeeded, 21 failed). The 18 
re-enabled files:
   
   decimalArithmeticOperations, 
having-and-order-by-recursive-type-name-resolution, ilike-all, ilike-any, 
like-all, like-any, order-by-and-having-on-top-of-aggregate-with-join, 
postgreSQL/date, random, replacing-missing-expression-with-alias, 
subquery/scalar-subquery/scalar-subquery-predicate, tablesample-negative, time, 
transform, type-coercion-edge-cases, typeCoercion/native/dateTimeOperations, 
typeCoercion/native/decimalPrecision, window-with-aggregates.
   
   Only test resources and `VeloxSQLQueryTestSettings` in `gluten-ut/spark43` 
change, so the Spark 4.2 and 3.5 modules are unaffected. `spotless:check` and 
scalastyle pass.
   
   ## Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Code (Claude Sonnet 5.5)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to