[ https://issues.apache.org/jira/browse/SPARK-41236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jingxiong zhong updated SPARK-41236: ------------------------------------ Description: `select collect_set(age) as age from db_table.table1 group by name having size(age) > 1 ` a simple sql, it work well in spark2.4, but doesn't work in spark3.2.0 Is it a bug or a new standard? like this: spark-sql> create db1.table1(age int, name string); Time taken: 1.709 seconds spark-sql> insert into db1.table1 values(1, 'a'); Time taken: 2.114 seconds spark-sql> insert into db1.table1 values(2, 'b'); Time taken: 10.208 seconds spark-sql> insert into db1.table1 values(3, 'c'); Time taken: 0.673 seconds spark-sql> select collect_set(age) as age > from db1.table1 > group by name > having size(age) > 1 ; was: `select collect_set(age) as age from db_table.table1 group by name having size(age) > 1 ` a simple sql, it work well in spark2.4, but doesn't work in spark3.2.0 Is it a bug or a new standard? like this: spark-sql> create db1.table1(age int, name string); Time taken: 1.709 seconds spark-sql> insert into db1.table1 values(1, 'a'); Time taken: 2.114 seconds spark-sql> insert into db1.table1 values(2, 'b'); Time taken: 10.208 seconds spark-sql> insert into db1.table1 values(3, 'c'); Time taken: 0.673 seconds spark-sql> select collect_set(age) as age > from db1.table1 > group by name > having size(age) > 1 ; Time taken: 3.022 seconds > The renamed field name cannot be recognized after group filtering > ----------------------------------------------------------------- > > Key: SPARK-41236 > URL: https://issues.apache.org/jira/browse/SPARK-41236 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.2.0 > Reporter: jingxiong zhong > Priority: Major > > `select collect_set(age) as age > from db_table.table1 > group by name > having size(age) > 1 ` > a simple sql, it work well in spark2.4, but doesn't work in spark3.2.0 > Is it a bug or a new standard? > like this: > spark-sql> create db1.table1(age int, name string); > Time taken: 1.709 seconds > spark-sql> insert into db1.table1 values(1, 'a'); > Time taken: 2.114 seconds > spark-sql> insert into db1.table1 values(2, 'b'); > Time taken: 10.208 seconds > spark-sql> insert into db1.table1 values(3, 'c'); > Time taken: 0.673 seconds > spark-sql> select collect_set(age) as age > > from db1.table1 > > group by name > > having size(age) > 1 ; -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org