Cheng Lian created SPARK-16291:
----------------------------------
Summary: Invalid aggregate functions like MAX(COUNT(*)) are not
captured by CheckAnalysis
Key: SPARK-16291
URL: https://issues.apache.org/jira/browse/SPARK-16291
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.2, 2.0.0
Reporter: Cheng Lian
Assignee: Cheng Lian
Priority: Minor
This issue can be reproduced by the following snippet:
{code}
spark.range(1).createOrReplaceTempView("test")
spark.sql("SELECT MAX(COUNT(*)) FROM test GROUP BY id")
{code}
An analysis phase error report like the following is expected:
bq. It is not allowed to use an aggregate function in the argument of another
aggregate function. ...
However, this error is not captured at analysis phase, but leads an execution
time exception:
bq. Error in SQL statement: SparkException: Job aborted due to stage failure:
Task 9 in stage 355.0 failed 4 times, most recent failure: Lost task 9.3 in
stage 355.0 (TID 12920, ip-10-31-224-146.ec2.internal):
java.lang.UnsupportedOperationException: Cannot evaluate expression:
(count(1),mode=Complete,isDistinct=false)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
