rdtr opened a new pull request, #13585: URL: https://github.com/apache/gluten/pull/13585
Depends on #13520. Only the last commit belongs to this PR: https://github.com/rdtr/incubator-gluten/commit/92431899136475ae2a91058d45a1cfca3e629c2f ## What changes were proposed in this pull request? Spark 4.3 (SPARK-58213) returns NULL from `corr` on a zero-variance column in ANSI mode, which is what Velox already does. Spark 4.2 and earlier throw `DIVIDE_BY_ZERO` there (#13255). - `MiscOperatorSuite` "Support multi-children count with row construct": ANSI mode is now enabled on Spark 4.3+ and still disabled before that. `ansiFallback` is turned off so the query is offloaded. - `VeloxAggregateFunctionsSuite`: new test "corr with zero variance column in ANSI mode", ANSI on, runs only on Spark 4.3+ (`assume(isSparkVersionGE("4.3"))`), checks the result against vanilla Spark and that `HashAggregateExecTransformer` is used. It is added here because `MiscOperatorSuite` is ignored on Spark 4.2+ (GLUTEN-13179), so the first change alone is not exercised on 4.3 yet. Spark 4.3.0 contains SPARK-58213 (`3a07a3dbb0e`), 4.2.1-rc3 does not. Relates to #13255. This covers the Spark 4.3 half of that issue; the 4.2-and-earlier behavior difference is still open, so I would keep the issue open. ## How was this patch tested? `mvn test -Pspark-4.3 -Pbackends-velox -pl backends-velox` with `VeloxAggregateFunctionsDefaultSuite`, "corr with zero variance column in ANSI mode": 1 passed. Without `ansiFallback=false` the same test fails because every operator falls back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
