gaborkaszab commented on code in PR #16125:
URL: https://github.com/apache/iceberg/pull/16125#discussion_r3843294466
##########
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java:
##########
@@ -1753,6 +1764,64 @@ public void testPartitionsTableDeleteStats() {
}
}
+ @Test
+ public void testPartitionsTableDvCount() {
+ TableIdentifier tableIdentifier = TableIdentifier.of("db",
"partitions_dv_test");
+ Table table =
+ createTable(
+ tableIdentifier, SCHEMA, SPEC,
ImmutableMap.of(TableProperties.FORMAT_VERSION, "3"));
+ Dataset<Row> df =
+ spark.createDataFrame(Lists.newArrayList(new SimpleRecord(1, "a")),
SimpleRecord.class);
+ df.select("id", "data")
+ .write()
+ .format("iceberg")
+ .mode("append")
+ .save(loadLocation(tableIdentifier));
+ table.refresh();
+ DataFile dataFile = TestHelpers.dataFiles(table).get(0);
+ DeleteFile dvFile = FileGenerationUtil.generateDV(table, dataFile);
+ table.newRowDelta().addDeletes(dvFile).commit();
+ table.refresh();
+
+ List<Row> actual =
+ spark
+ .read()
+ .format("iceberg")
+ .load(loadLocation(tableIdentifier, "partitions"))
+ .collectAsList();
+
+ assertThat(actual).hasSize(1);
+ assertThat((int) actual.get(0).getAs("dv_count")).isEqualTo(1);
+
+ // add a second data file and DV to test dv_count = 2
Review Comment:
Not entirely sure, but I recall I commented earlier that such an "add
another file and dv" step probably doesn't add much to test coverage. This test
could be:
1) set up table with 2 files and a DV for each
2) query partitions table and verify 2 DVs
No need to split this into 2 steps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
