MaxGekk commented on code in PR #45622:
URL: https://github.com/apache/spark/pull/45622#discussion_r1533260149
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala:
##########
@@ -295,10 +295,50 @@ class AnalysisErrorSuite extends AnalysisTest with
DataTypeErrorsBase {
"aggregate(array(1, 2, 3), 0, (acc, x) -> acc + x) FILTER (WHERE c >
1)", 7, 76)))
}
- errorTest(
- "non-deterministic filter predicate in aggregate functions",
- CatalystSqlParser.parsePlan("SELECT count(a) FILTER (WHERE rand(int(c)) >
1) FROM TaBlE2"),
- "FILTER expression is non-deterministic, it cannot be used in aggregate
functions" :: Nil)
+ test("SPARK-47256: non deterministic FILTER expression in an aggregate
function") {
+ val plan =
+ CatalystSqlParser.parsePlan("SELECT count(a) FILTER (WHERE rand(int(c))
> 1) FROM TaBlE2")
Review Comment:
@davidm-db Could you reproduce the errors using public APIs, please. If it
is not possible, we should consider to convert the error to internal errors
(see `SparkException.internalError()`).
##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -4627,6 +4627,26 @@
],
"sqlState" : "42KDF"
},
+ "AGGREGATE_FILTER_EXPRESSION_NON_DETERMINISTIC" : {
Review Comment:
Please, create an error class w/ a common error message, and its sub-classes
w/ specific error messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
