kbendick commented on code in PR #4847:
URL: https://github.com/apache/iceberg/pull/4847#discussion_r907938573
##########
spark/v2.4/spark/src/test/java/org/apache/iceberg/spark/source/TestIcebergSourceTablesBase.java:
##########
@@ -991,18 +1005,34 @@ public void testAllManifestsTable() {
Table manifestTable = loadTable(tableIdentifier, "all_manifests");
Dataset<Row> df1 = spark.createDataFrame(Lists.newArrayList(new
SimpleRecord(1, "a")), SimpleRecord.class);
- List<ManifestFile> manifests = Lists.newArrayList();
-
df1.select("id", "data").write()
.format("iceberg")
.mode("append")
.save(loadLocation(tableIdentifier));
- manifests.addAll(table.currentSnapshot().allManifests());
+ table.updateProperties()
+ .set(TableProperties.FORMAT_VERSION, "2")
+ .commit();
+
+ DataFile dataFile =
Iterables.getFirst(table.currentSnapshot().addedFiles(table.io()), null);
+ PartitionSpec dataFileSpec = table.specs().get(dataFile.specId());
+ StructLike dataFilePartition = dataFile.partition();
+
+ PositionDelete<InternalRow> delete = PositionDelete.create();
+ delete.set(dataFile.path(), 0L, null);
+
+ DeleteFile deleteFile = writePositionDeletes(table, dataFileSpec,
dataFilePartition, ImmutableList.of(delete));
+
+ table.newRowDelta()
+ .addDeletes(deleteFile)
+ .commit();
table.newDelete().deleteFromRowFilter(Expressions.alwaysTrue()).commit();
- manifests.addAll(table.currentSnapshot().allManifests());
+ Stream<Pair<Long, ManifestFile>> snapshotIdToManifests =
+ StreamSupport.stream(table.snapshots().spliterator(), false)
Review Comment:
Nit: Can you add an inline comment for what the `false` stands for? If its
something like parallel or something then don’t worry about it, but usually it
helps me reading the code to have the parameter name in-lined like `, /*
caseSensitive */ false);`.
But again, thinking on it (and the usage of spliterator), if it’s parallel
or something then feel free to ignore it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
