[
https://issues.apache.org/jira/browse/SPARK-41236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jingxiong zhong updated SPARK-41236:
------------------------------------
Description:
`select collect_set(age) as age
from db_table.table1
group by name
having size(age) > 1 `
a simple sql, it work well in spark2.4, but doesn't work in spark3.2.0
Is it a bug or a new standard?
like this:
spark-sql> create db1.table1(age int, name string);
Time taken: 1.709 seconds
spark-sql> insert into db1.table1 values(1, 'a');
Time taken: 2.114 seconds
spark-sql> insert into db1.table1 values(2, 'b');
Time taken: 10.208 seconds
spark-sql> insert into db1.table1 values(3, 'c');
Time taken: 0.673 seconds
spark-sql> select collect_set(age) as age
> from db1.table1
> group by name
> having size(age) > 1 ;
Time taken: 3.022 seconds
was:
`select collect_set(age) as age
from db_table.table1
group by name
having size(age) > 1 `
a simple sql, it work well in spark2.4, but doesn't work in spark3.2.0
Is it a bug or a new standard?
> The renamed field name cannot be recognized after group filtering
> -----------------------------------------------------------------
>
> Key: SPARK-41236
> URL: https://issues.apache.org/jira/browse/SPARK-41236
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: jingxiong zhong
> Priority: Major
>
> `select collect_set(age) as age
> from db_table.table1
> group by name
> having size(age) > 1 `
> a simple sql, it work well in spark2.4, but doesn't work in spark3.2.0
> Is it a bug or a new standard?
> like this:
> spark-sql> create db1.table1(age int, name string);
> Time taken: 1.709 seconds
> spark-sql> insert into db1.table1 values(1, 'a');
> Time taken: 2.114 seconds
> spark-sql> insert into db1.table1 values(2, 'b');
> Time taken: 10.208 seconds
> spark-sql> insert into db1.table1 values(3, 'c');
> Time taken: 0.673 seconds
> spark-sql> select collect_set(age) as age
> > from db1.table1
> > group by name
> > having size(age) > 1 ;
> Time taken: 3.022 seconds
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
