davidm-db commented on code in PR #45622:
URL: https://github.com/apache/spark/pull/45622#discussion_r1533667336
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala:
##########
@@ -295,10 +295,50 @@ class AnalysisErrorSuite extends AnalysisTest with
DataTypeErrorsBase {
"aggregate(array(1, 2, 3), 0, (acc, x) -> acc + x) FILTER (WHERE c >
1)", 7, 76)))
}
- errorTest(
- "non-deterministic filter predicate in aggregate functions",
- CatalystSqlParser.parsePlan("SELECT count(a) FILTER (WHERE rand(int(c)) >
1) FROM TaBlE2"),
- "FILTER expression is non-deterministic, it cannot be used in aggregate
functions" :: Nil)
+ test("SPARK-47256: non deterministic FILTER expression in an aggregate
function") {
+ val plan =
+ CatalystSqlParser.parsePlan("SELECT count(a) FILTER (WHERE rand(int(c))
> 1) FROM TaBlE2")
Review Comment:
All errors are available/visible through public API. So, if I understood
your comment properly, I don't think this should be converted to internal
errors. Example of one of the errors:
<img width="1011" alt="image"
src="https://github.com/apache/spark/assets/163021185/0c117a62-98c1-42b7-80fc-09e8309d918f";>
However, triggering public API in this test suite seems like too much setup
work, since this suite doesn't extend `SQLTestUtilsBase` trait, that provides
the support for `sql()` method that triggers public API.
What I'm thinking about is additional tests in some other test suite that
would reproduce errors through public API, though I'm not yet sure where
exactly to add such tests.
Am I missing something or my thoughts sound good?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
