szehon-ho commented on code in PR #4898:
URL: https://github.com/apache/iceberg/pull/4898#discussion_r1185706910
##########
core/src/test/java/org/apache/iceberg/TestManifestReader.java:
##########
@@ -142,9 +147,61 @@ public void testDeleteFilePositions() throws IOException {
long expectedPos = 0L;
for (DeleteFile file : reader) {
Assert.assertEquals("Position should match", (Long) expectedPos,
file.pos());
- Assert.assertEquals("Position from field index should match",
expectedPos, ((BaseFile) file).get(17));
+ Assert.assertEquals("Position from field index should match",
expectedPos, ((BaseFile) file).get(18));
expectedPos += 1;
}
}
}
+
+ @Test
+ public void testManifestCompatible() throws IOException {
+ Schema schema = new Schema(
+ required(1, "id", Types.LongType.get()),
+ required(2, "timestamp", Types.TimestampType.withZone()),
+ required(3, "category", Types.StringType.get()),
+ required(4, "data", Types.StringType.get()),
+ required(5, "double", Types.DoubleType.get()));
+
+ PartitionSpec spec = PartitionSpec.builderFor(schema)
+ .identity("category")
+ .hour("timestamp")
+ .bucket("id", 16)
+ .build();
+ PartitionData partition = DataFiles.data(spec,
"category=cheesy/timestamp_hour=10/id_bucket=3");
+
+ String path =
"s3://bucket/table/category=cheesy/timestamp_hour=10/id_bucket=3/file.avro";
+ long fileSize = 150972L;
+ FileFormat format = FileFormat.AVRO;
+
+ Metrics metrics = new Metrics(
+ 1587L,
+ ImmutableMap.of(1, 15L, 2, 122L, 3, 4021L, 4, 9411L, 5, 15L), // sizes
+ ImmutableMap.of(1, 100L, 2, 100L, 3, 100L, 4, 100L, 5, 100L), //
value counts
+ ImmutableMap.of(1, 0L, 2, 0L, 3, 0L, 4, 0L, 5, 0L), // null value
counts
+ ImmutableMap.of(5, 10L), // nan value counts
+ ImmutableMap.of(1, Conversions.toByteBuffer(Types.IntegerType.get(),
1)), // lower bounds
+ ImmutableMap.of(1, Conversions.toByteBuffer(Types.IntegerType.get(),
1))); // upper bounds
+ Integer sortOrderId = 2;
+
+ String fileName = String.format("OldManifestFileV%s.avro", formatVersion);
Review Comment:
Yea I was thinking to make a Test version of those writers.. not sure if
its possible. Anyway, I guess this works too, only issue is wont be too
debuggable if something goes wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
