cloud-fan commented on code in PR #58077: URL: https://github.com/apache/spark/pull/58077#discussion_r3807100136
########## sql/core/src/main/scala/org/apache/spark/sql/execution/subquery.scala: ########## @@ -125,7 +125,10 @@ case class InSubqueryExec( @transient private lazy val inSet = InSet(child, result.toSet) - override def nullable: Boolean = child.nullable + // x IN (subquery) evaluates to UNKNOWN when no match is found and the subquery result + // contains NULL, so nullable must account for whether the subquery output can produce NULLs, + // not just whether the child expression is nullable. See SPARK-58481. + override def nullable: Boolean = child.nullable || plan.output.head.nullable Review Comment: **Blocking:** Please preserve UNKNOWN semantics for every subquery output here. A multi-column `IN` can have a non-nullable first output and a nullable later output, so checking only `head` still declares this expression non-nullable; those results are struct rows, and `InSet.hasNull` also misses nested NULL fields. Please propagate nested-field UNKNOWN at runtime, mirror the logical all-output nullability contract (including the legacy config), and add a focused multi-column regression. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
