rdblue commented on a change in pull request #3365:
URL: https://github.com/apache/iceberg/pull/3365#discussion_r736032154
##########
File path: flink/src/test/java/org/apache/iceberg/flink/SimpleDataUtil.java
##########
@@ -267,17 +268,28 @@ public static StructLikeSet actualRowSet(Table table,
Long snapshotId, String...
public static Map<Long, List<DataFile>> snapshotToDataFiles(Table table)
throws IOException {
table.refresh();
+
Map<Long, List<DataFile>> result = Maps.newHashMap();
- List<ManifestFile> manifestFiles = table.currentSnapshot().dataManifests();
- for (ManifestFile manifestFile : manifestFiles) {
- try (ManifestReader<DataFile> reader = ManifestFiles.read(manifestFile,
table.io())) {
- List<DataFile> dataFiles = Lists.newArrayList(reader);
- if (result.containsKey(manifestFile.snapshotId())) {
- result.get(manifestFile.snapshotId()).addAll(dataFiles);
- } else {
- result.put(manifestFile.snapshotId(), dataFiles);
- }
+ Snapshot current = table.currentSnapshot();
Review comment:
The reason why we don't expose the manifest entry is that it's
confusing. We don't want users to read a manifest and assume that all of the
entries represent files in the table because we track deleted files in the same
metadata. So it is more that users would need to know more about the spec and
we don't think that it is likely to be used correctly.
I'm still open to the idea of making this public. But if we don't need it
then I'd opt not to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
