yyanyy commented on a change in pull request #3320:
URL: https://github.com/apache/iceberg/pull/3320#discussion_r732344704
##########
File path: orc/src/main/java/org/apache/iceberg/orc/OrcMetrics.java
##########
@@ -196,11 +196,17 @@ private static Metrics buildOrcMetrics(final long
numOfRows, final TypeDescripti
min = Math.toIntExact((long) min);
}
} else if (columnStats instanceof DoubleColumnStatistics) {
- // since Orc includes NaN for upper/lower bounds of floating point
columns, and we don't want this behavior,
- // we have tracked metrics for such columns ourselves and thus do not
need to rely on Orc's column statistics.
- Preconditions.checkNotNull(fieldMetrics,
- "[BUG] Float or double type columns should have metrics being
tracked by Iceberg Orc writers");
- min = fieldMetrics.lowerBound();
+ if (fieldMetrics != null) {
+ // since Orc includes NaN for upper/lower bounds of floating point
columns, and we don't want this behavior,
+ // we have tracked metrics for such columns ourselves and thus do not
need to rely on Orc's column statistics.
+ min = fieldMetrics.lowerBound();
+ } else {
+ // imported files will not have metrics that were tracked by Iceberg,
so fall back to the file's metrics.
+ min = ((DoubleColumnStatistics) columnStats).getMinimum();
+ if (type.typeId() == Type.TypeID.FLOAT) {
+ min = ((Double) min).floatValue();
Review comment:
wondering if we want to reassign min to negative infinite if the this
value tracked by ORC is NaN
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
