kbendick commented on a change in pull request #3332:
URL: https://github.com/apache/iceberg/pull/3332#discussion_r733149334
##########
File path: orc/src/main/java/org/apache/iceberg/orc/OrcMetrics.java
##########
@@ -234,11 +240,17 @@ private static Metrics buildOrcMetrics(final long
numOfRows, final TypeDescripti
max = Math.toIntExact((long) max);
}
} else if (columnStats instanceof DoubleColumnStatistics) {
- // since Orc includes NaN for upper/lower bounds of floating point
columns, and we don't want this behavior,
- // we have tracked metrics for such columns ourselves and thus do not
need to rely on Orc's column statistics.
- Preconditions.checkNotNull(fieldMetrics,
- "[BUG] Float or double type columns should have metrics being
tracked by Iceberg Orc writers");
- max = fieldMetrics.upperBound();
+ if (fieldMetrics != null) {
+ // since Orc includes NaN for upper/lower bounds of floating point
columns, and we don't want this behavior,
+ // we have tracked metrics for such columns ourselves and thus do not
need to rely on Orc's column statistics.
+ max = fieldMetrics.upperBound();
+ } else {
+ // imported files will not have metrics that were tracked by Iceberg,
so fall back to the file's metrics.
+ max = ((DoubleColumnStatistics) columnStats).getMaximum();
+ if (type.typeId() == Type.TypeID.FLOAT) {
+ max = ((Double) max).floatValue();
+ }
+ }
Review comment:
I tested without this fix and encountered the error, so I'm sure that
the test does in fact hit the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
