Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/19523#discussion_r145307826
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala
---
@@ -102,7 +102,8 @@ case class InMemoryTableScanExec(
case IsNull(a: Attribute) => statsFor(a).nullCount > 0
case IsNotNull(a: Attribute) => statsFor(a).count -
statsFor(a).nullCount > 0
- case In(_: AttributeReference, list: Seq[Expression]) if list.isEmpty
=> Literal.FalseLiteral
+ // We rely on the optimizations in
org.apache.spark.sql.catalyst.optimizer.OptimizeIn
+ // to be sure that the list cannot be empty
--- End diff --
IMHO this comment is not accurate, since in optimizer we only deal with the
case attribute is not nullable.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
