Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21882#discussion_r205508221
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala
---
@@ -183,6 +183,13 @@ case class InMemoryTableScanExec(
private val stats = relation.partitionStatistics
private def statsFor(a: Attribute) = stats.forAttribute(a)
+ // For some ColumnStats, for instance, ObjectColumnStats always has
nulls for lower and upper
+ // bounds.
+ private def nullSafeEval(
+ attr: AttributeReference)(func: AttributeReference => Expression):
Expression = {
+ attr.isNull || func(attr)
--- End diff --
Thanks, @cloud-fan.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
