rdblue commented on code in PR #16867:
URL: https://github.com/apache/iceberg/pull/16867#discussion_r3456164024
##########
core/src/main/java/org/apache/iceberg/TrackedFileAdapters.java:
##########
@@ -405,6 +418,135 @@ public DeleteFile copyWithStats(Set<Integer>
requestedColumnIds) {
}
}
+ /**
+ * Adapts a TrackedFile DATA_MANIFEST or DELETE_MANIFEST entry to the {@link
ManifestFile}
+ * interface.
+ */
+ private static class TrackedManifestFile implements ManifestFile {
+ private final TrackedFile file;
+
+ private TrackedManifestFile(TrackedFile file) {
+ Preconditions.checkArgument(
+ file.tracking().dataSequenceNumber() != null,
+ "Cannot create manifest file: no data sequence number");
+ Preconditions.checkArgument(
+ file.tracking().snapshotId() != null, "Cannot create manifest file:
no snapshot ID");
+ Preconditions.checkArgument(
+ file.manifestInfo() != null, "Cannot create manifest file: no
manifest info");
+ Preconditions.checkArgument(
+ file.manifestInfo().dv() == null,
+ "Cannot adapt manifest with a deletion vector to ManifestFile: %s",
+ file.location());
+ this.file = file;
+ }
+
+ @Override
+ public String path() {
+ return file.location();
+ }
+
+ @Override
+ public long length() {
+ return file.fileSizeInBytes();
+ }
+
+ @Override
+ public int partitionSpecId() {
+ throw new UnsupportedOperationException(
+ "Tracked manifests are not bound to a single partition spec");
+ }
+
+ @Override
+ public ManifestContent content() {
+ return file.contentType() == FileContent.DATA_MANIFEST
+ ? ManifestContent.DATA
+ : ManifestContent.DELETES;
+ }
+
+ @Override
+ public long sequenceNumber() {
+ return file.tracking().dataSequenceNumber();
+ }
+
+ @Override
+ public long minSequenceNumber() {
+ return file.manifestInfo().minSequenceNumber();
+ }
+
+ @Override
+ public Long snapshotId() {
+ return file.tracking().snapshotId();
+ }
+
+ @Override
+ public Integer addedFilesCount() {
+ return file.manifestInfo().addedFilesCount();
+ }
+
+ @Override
+ public Long addedRowsCount() {
+ return file.manifestInfo().addedRowsCount();
+ }
+
+ @Override
+ public Integer existingFilesCount() {
+ return file.manifestInfo().existingFilesCount();
+ }
+
+ @Override
+ public Long existingRowsCount() {
+ return file.manifestInfo().existingRowsCount();
+ }
+
+ @Override
+ public Integer deletedFilesCount() {
+ return file.manifestInfo().deletedFilesCount();
+ }
+
+ @Override
+ public Long deletedRowsCount() {
+ return file.manifestInfo().deletedRowsCount();
+ }
+
+ // v4 does not store partition summaries on manifests, so return null.
+ @Override
+ public List<PartitionFieldSummary> partitions() {
+ return null;
+ }
+
+ @Override
+ public ByteBuffer keyMetadata() {
+ return file.keyMetadata();
+ }
+
+ @Override
+ public Long firstRowId() {
+ return file.tracking().firstRowId();
+ }
+
+ @Override
+ public ManifestFile copy() {
+ return new TrackedManifestFile(file.copy());
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (this == other) {
+ return true;
+ } else if (!(other instanceof TrackedManifestFile)) {
Review Comment:
I don't think that the implementation of `GenericManifestFile` is very
helpful. It considers two files equal if they have the same path (which I think
is reasonable) but it also excludes any implementation other than
`GenericManifestFile`, which means that they aren't actually interchangeable as
the interface suggests.
In addition, this creates a situation where `a.equals(b)` may be true, but
`b.equals(a)` can be false for reasonable implementations.
I'd prefer to remove this and fix places where it is assumed or used. Good
catch, @gaborkaszab!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
