AngersZhuuuu commented on issue #26406: [WIP][SPARK-29145][SQL][FOLLOW-UP] Port in-subquery-on-join-condition.sql URL: https://github.com/apache/spark/pull/26406#issuecomment-550196498 > Ur.... I see. You found the bug caused by your previous pr, right? If so, could you file a new jira for that and investigate the root cause of the bug there? I don't think it is caused by my pr. My pr focus on IN/NOT IN, behavior between `IN/NOT IN` and `Exists/NOT EXISTS ` is different. Run this example in 2.4.0 ``` == Parsed Logical Plan == 'Project ['s1.id, 's2.id AS id2#4] +- 'Join LeftOuter, (('s1.id = 's2.id) && exists#3 []) : +- 'Project [*] : +- 'Filter ('s3.id > 6) : +- 'UnresolvedRelation `s3` :- 'UnresolvedRelation `s1` +- 'UnresolvedRelation `s2` == Analyzed Logical Plan == org.apache.spark.sql.AnalysisException: Table or view not found: `s3`; line 3 pos 27; 'Project ['s1.id, 's2.id AS id2#4] +- 'Join LeftOuter, ((id#0 = id#1) && exists#3 []) : +- 'Project [*] : +- 'Filter ('s3.id > 6) : +- 'UnresolvedRelation `s3` :- SubqueryAlias `s1` : +- Project [id#0] : +- SubqueryAlias `s1` : +- LocalRelation [id#0] +- SubqueryAlias `s2` +- Project [id#1] +- SubqueryAlias `s2` +- LocalRelation [id#1] org.apache.spark.sql.AnalysisException: Table or view not found: `s3`; line 3 pos 27; 'Project ['s1.id, 's2.id AS id2#4] +- 'Join LeftOuter, ((id#0 = id#1) && exists#3 []) : +- 'Project [*] : +- 'Filter ('s3.id > 6) : +- 'UnresolvedRelation `s3` :- SubqueryAlias `s1` : +- Project [id#0] : +- SubqueryAlias `s1` : +- LocalRelation [id#0] +- SubqueryAlias `s2` +- Project [id#1] +- SubqueryAlias `s2` +- LocalRelation [id#1] == Optimized Logical Plan == org.apache.spark.sql.AnalysisException: Table or view not found: `s3`; line 3 pos 27; 'Project ['s1.id, 's2.id AS id2#4] +- 'Join LeftOuter, ((id#0 = id#1) && exists#3 []) : +- 'Project [*] : +- 'Filter ('s3.id > 6) : +- 'UnresolvedRelation `s3` :- SubqueryAlias `s1` : +- Project [id#0] : +- SubqueryAlias `s1` : +- LocalRelation [id#0] +- SubqueryAlias `s2` +- Project [id#1] +- SubqueryAlias `s2` +- LocalRelation [id#1] == Physical Plan == org.apache.spark.sql.AnalysisException: Table or view not found: `s3`; line 3 pos 27; 'Project ['s1.id, 's2.id AS id2#4] +- 'Join LeftOuter, ((id#0 = id#1) && exists#3 []) : +- 'Project [*] : +- 'Filter ('s3.id > 6) : +- 'UnresolvedRelation `s3` :- SubqueryAlias `s1` : +- Project [id#0] : +- SubqueryAlias `s1` : +- LocalRelation [id#0] +- SubqueryAlias `s2` +- Project [id#1] +- SubqueryAlias `s2` +- LocalRelation [id#1] Time taken: 1.455 seconds, Fetched 1 row(s) ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
