Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/19479#discussion_r147886882
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala
---
@@ -177,13 +180,12 @@ object ColumnStat extends Logging {
Some(ColumnStat(
distinctCount = BigInt(map(KEY_DISTINCT_COUNT).toLong),
// Note that flatMap(Option.apply) turns Option(null) into None.
- min = map.get(KEY_MIN_VALUE)
- .map(fromExternalString(_, field.name,
field.dataType)).flatMap(Option.apply),
- max = map.get(KEY_MAX_VALUE)
- .map(fromExternalString(_, field.name,
field.dataType)).flatMap(Option.apply),
+ min = map.get(KEY_MIN_VALUE).map(fromString(_, field.name,
field.dataType)),
+ max = map.get(KEY_MAX_VALUE).map(fromString(_, field.name,
field.dataType)),
--- End diff --
Yea, but I tend to revert the change because keep `flatMap(Option.apply)`
is more robust.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
