Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/19831#discussion_r154245570
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -418,7 +418,7 @@ private[hive] class HiveClientImpl(
// Note that this statistics could be overridden by Spark's
statistics if that's available.
val totalSize =
properties.get(StatsSetupConst.TOTAL_SIZE).map(BigInt(_))
val rawDataSize =
properties.get(StatsSetupConst.RAW_DATA_SIZE).map(BigInt(_))
- val rowCount =
properties.get(StatsSetupConst.ROW_COUNT).map(BigInt(_)).filter(_ >= 0)
+ val rowCount =
properties.get(StatsSetupConst.ROW_COUNT).map(BigInt(_)).filter(_ > 0)
--- End diff --
Maybe this could be more clear:
```scala
val rowCount = properties.get(StatsSetupConst.ROW_COUNT).map(BigInt(_))
val stats =
if (totalSize.isDefined && totalSize.get > 0L) {
Some(CatalogStatistics(sizeInBytes = totalSize.get, rowCount =
rowCount.filter(_ > 0)))
} else if (rawDataSize.isDefined && rawDataSize.get > 0) {
Some(CatalogStatistics(sizeInBytes = rawDataSize.get, rowCount =
rowCount.filter(_ > 0)))
} else {
None
}
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
