Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19522#discussion_r145267637
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala
---
@@ -102,7 +102,8 @@ case class InMemoryTableScanExec(
case IsNull(a: Attribute) => statsFor(a).nullCount > 0
case IsNotNull(a: Attribute) => statsFor(a).count -
statsFor(a).nullCount > 0
- case In(_: AttributeReference, list: Seq[Expression]) if list.isEmpty
=> Literal.FalseLiteral
+ // We rely on the optimizations in
org.apache.spark.sql.catalyst.optimizer.OptimizeIn
--- End diff --
We should not rely on Optimizer for fixing the bugs.
We need to fix the line 107 anyway.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
