Jacek Laskowski created SPARK-20122:
---------------------------------------
Summary: Analyzer reports "unresolved operator 'Aggregate" for
nonexistent window specifications
Key: SPARK-20122
URL: https://issues.apache.org/jira/browse/SPARK-20122
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.2.0
Reporter: Jacek Laskowski
Priority: Minor
{{Analyzer}} should be more elaborate on the root cause of the
{{org.apache.spark.sql.AnalysisException}} when {{WindowSpecReference}} cannot
be resolved.
I think the real issue is that {{AstBuilder}} creates a lookup table for
{{WindowSpecDefinition}}s per name and when it cannot resolve
{{WindowSpecReference}} it simply fails analysis (it's too early in the SQL
parsing).
{code}
scala> table.selectExpr("count() OVER nonexistentWindowSpec")
org.apache.spark.sql.AnalysisException: unresolved operator 'Aggregate
[unresolvedwindowexpression(count(),
WindowSpecReference(nonexistentWindowSpec)) AS
unresolvedwindowexpression(count())#15];;
'Aggregate [unresolvedwindowexpression(count(),
WindowSpecReference(nonexistentWindowSpec)) AS
unresolvedwindowexpression(count())#15]
+- LocalRelation <empty>, [value#11]
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:40)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:90)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$3.apply(CheckAnalysis.scala:408)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$3.apply(CheckAnalysis.scala:407)
at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:127)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:407)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:90)
at
org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:53)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:67)
at
org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$withPlan(Dataset.scala:2832)
at org.apache.spark.sql.Dataset.select(Dataset.scala:1137)
at org.apache.spark.sql.Dataset.selectExpr(Dataset.scala:1172)
... 48 elided
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
