Su Qilong created SPARK-33837:
---------------------------------
Summary: Correlated subquery field resolve bug when inner table
has a field with the same name with outer table
Key: SPARK-33837
URL: https://issues.apache.org/jira/browse/SPARK-33837
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.3
Reporter: Su Qilong
For the following statements:
{code:java}
create table t1(id int, name string)
create table id(key int, name string);
select * from id where name in (select name from t1 where t1.id = id.key)
{code}
This query is supported by spark, but since table t1 has an attribute named id,
spark will raise an error like:
{noformat}
Can't extract value from id#83: need struct type but got int;
{noformat}
According to implementation in Spark's ResolveReference rule, the name
resolving precedence is `db.table` => `table.attr` => `attr.innerfield`
So here id.key should be resolved as table id's attribute, rather than the
innerfield of the attribute of table t1
The problem in this resolving bug lies in the subquery resolving. In
ResolveSubquery rule, we first try to resolve subquery independently, and only
when there're unresolved attribute in subquery, we try resolveOuterReferences.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
