Dilip Biswal created SPARK-20334:
------------------------------------
Summary: Return a better error message when correlated predicates
contain aggregate expression that has mixture of outer and local references
Key: SPARK-20334
URL: https://issues.apache.org/jira/browse/SPARK-20334
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 2.1.0
Reporter: Dilip Biswal
Priority: Minor
Currently subqueries with correlated predicates containing aggregate expression
having mixture of outer references and local references generate a code gen
error like following :
{code:java}
Cannot evaluate expression: min((input[0, int, false] + input[4, int, false]))
at
org.apache.spark.sql.catalyst.expressions.Unevaluable$class.doGenCode(Expression.scala:226)
at
org.apache.spark.sql.catalyst.expressions.aggregate.AggregateExpression.doGenCode(interfaces.scala:87)
at
org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:106)
at
org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:103)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:103)
at
org.apache.spark.sql.catalyst.expressions.BinaryExpression.nullSafeCodeGen(Expression.scala:461)
at
org.apache.spark.sql.catalyst.expressions.BinaryExpression.defineCodeGen(Expression.scala:443)
at
org.apache.spark.sql.catalyst.expressions.BinaryComparison.doGenCode(predicates.scala:431)
at
org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:106)
at
org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:103)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:103)
{code}
We should catch this situation and return a better error message to the user.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
