edgarRd commented on a change in pull request #199:
URL: https://github.com/apache/iceberg/pull/199#discussion_r433639447
##########
File path: core/src/test/java/org/apache/iceberg/TestMetrics.java
##########
@@ -206,13 +214,25 @@ public void testMetricsForTopLevelFields() throws
IOException {
assertCounts(9, 2L, 0L, metrics);
assertBounds(9, TimeType.get(), 2000L, 3000L, metrics);
assertCounts(10, 2L, 0L, metrics);
- assertBounds(10, TimestampType.withoutZone(), 0L, 1000L, metrics);
+ if (fileFormat() == FileFormat.ORC) {
+ // ORC-611: ORC only supports millisecond precision, so we adjust by 1
millisecond
+ assertBounds(10, TimestampType.withoutZone(), -1000L, 1000L, metrics);
+ } else {
+ assertBounds(10, TimestampType.withoutZone(), 0L, 900L, metrics);
+ }
assertCounts(11, 2L, 0L, metrics);
assertBounds(11, FixedType.ofLength(4),
ByteBuffer.wrap(fixed), ByteBuffer.wrap(fixed), metrics);
assertCounts(12, 2L, 0L, metrics);
assertBounds(12, BinaryType.get(),
ByteBuffer.wrap("S".getBytes()), ByteBuffer.wrap("W".getBytes()),
metrics);
+ if (fileFormat() == FileFormat.ORC) {
+ // TODO: enable when ORC-342 is fixed - ORC-342: creates inaccurate
timestamp/stats below epoch
+ // ORC-611: ORC only supports millisecond precision, so we adjust by 1
millisecond
+ // assertBounds(13, TimestampType.withoutZone(), -1000L, 1000L, metrics);
Review comment:
Due to [ORC-342](https://issues.apache.org/jira/browse/ORC-342) it's not
clear whether values in the range `[1970-01-01 00:00:00.000,1970-01-01
00:00:00.999]` were actually positive or negative. So, we can't actually tell
in this case whether to include bounds or not based on that condition since the
values themselves are wrong.
For any other values outside those ranges the metrics seem to be fine and it
looks like [ORC does adjust by 1 ms below
epoch](https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/impl/writer/TimestampTreeWriter.java#L136).
I've adjusted the test to check for this case, with a note on ORC-342.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
