Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21403#discussion_r206436489
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1422,11 +1422,26 @@ class Analyzer(
resolveSubQuery(s, plans)(ScalarSubquery(_, _, exprId))
case e @ Exists(sub, _, exprId) if !sub.resolved =>
resolveSubQuery(e, plans)(Exists(_, _, exprId))
- case In(value, Seq(l @ ListQuery(sub, _, exprId, _))) if
value.resolved && !l.resolved =>
+ case In(values, Seq(l @ ListQuery(_, _, exprId, _)))
+ if values.forall(_.resolved) && !l.resolved =>
val expr = resolveSubQuery(l, plans)((plan, exprs) => {
ListQuery(plan, exprs, exprId, plan.output)
})
- In(value, Seq(expr))
+ val subqueryOutput = expr.plan.output
+ val resolvedIn = In(values, Seq(expr))
+ if (values.length != subqueryOutput.length) {
+ throw new AnalysisException(
--- End diff --
thanks for your review @dilipbiswal.
> The right hand side columns looks confusing. Should we only display the
value exprs or the name exprs instead of both ?
I don't think so honestly. This is just the `sql` value of a named_struct.
Having a custom representation of it only for this use case doesn't seem a good
idea to me.
> perhaps we can take out the dot at the end of #columns .. and the
following lines ?
sure, I'll do in my next commit, thanks.
> We have this check in checkInputDataTypes and here ? Is there a way we
can have the number of input check in one place ?
I added the check here in order to avoid to waste time going on in the
analysis while we already know that it is going to fail. I haven't removed the
check from `checkInputDataTypes`, as I prefer staying on the safe side with an
additional check, but that is not met anymore, as we check it here.
> Just a question, should we have been able to do a type promotion here ?
I have not changed type promotion in this PR. The same behavior happens
before and after this PR. I think this can be proposed as a followup/new JIRA.
Thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
