JacobZheng created SPARK-45601:
----------------------------------
Summary: stackoverflow when executing rule ExtractWindowExpressions
Key: SPARK-45601
URL: https://issues.apache.org/jira/browse/SPARK-45601
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.2.3
Reporter: JacobZheng
I am encountering stackoverflow errors while executing the following test case.
I looked at the source code and it is ExtractWindowExpressions not extracting
the window correctly and encountering a dead loop at
resolveOperatorsDownWithPruning that is causing it.
{code:scala}
// Some comments here
test("agg filter contains window") {
val src = Seq((1, "b", "c")).toDF("col1", "col2", "col3")
.withColumn("test",
expr("count(col1) filter (where min(col1) over(partition by col2 order
by col3)>1)"))
src.show()
}
{code}
Now my question is this kind of in agg filter (window) is the correct usage? Or
should I add a check like spark sql and throw an error "It is not allowed to
use window functions inside WHERE clause"?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
