Bogdan Raducanu created SPARK-23316:

             Summary: AnalysisException after max iteration reached for IN query
                 Key: SPARK-23316
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.3.0
            Reporter: Bogdan Raducanu

Query to reproduce:
spark.range(10).where("(id,id) in (select id, null from range(3))").show

18/02/02 11:32:31 WARN BaseSessionStateBuilder$$anon$1: Max iterations (100) 
reached for batch Resolution
org.apache.spark.sql.AnalysisException: cannot resolve '(named_struct('id', 
`id`, 'id', `id`) IN (listquery()))' due to data type mismatch:
The data type of one or more elements in the left hand side of an IN subquery
is not compatible with the data type of the output of the subquery
Mismatched columns:
Left side:
[bigint, bigint].
Right side:
[bigint, bigint].;;

The error message includes the last plan which contains ~100 useless Projects.
Does not happen in branch-2.2.
It has something to do with TypeCoercion, it is doing a futile attempt  to 
change nullability.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to