rdblue commented on code in PR #18816:
URL: https://github.com/apache/iceberg/pull/18816#discussion_r4622661193
##########
core/src/main/java/org/apache/iceberg/V4ManifestReader.java:
##########
@@ -480,7 +604,44 @@ V4ManifestReader build() {
partitionProjections(),
requestedStatsFieldIds,
isUncommitted,
- scanMetrics);
+ scanMetrics,
+ changes,
+ deletedReference,
+ changes ? resolveChangeSnapshotId(tracking) : 0L,
+ tracking != null ? readBitmap(tracking.deletedPositions()) : null,
+ tracking != null ? readBitmap(tracking.replacedPositions()) : null);
+ }
+
+ private long resolveChangeSnapshotId(Tracking tracking) {
+ if (tracking == null) {
+ return manifest.snapshotId();
+ }
+
+ EntryStatus status = tracking.status();
+ Preconditions.checkArgument(
Review Comment:
I understand what you want to do here, but I don't think throwing an
exception is the right solution. This checks whether the manifest was changed.
But in the case above where tracking is null, we just assume that it changed
and get the changes from when it was added, deleted, or replaced.
Let's first take the case where you have a manifest with no tracking. If you
have a `TrackedFile` adapted to `ManifestFile` that has no tracking, then
`manifest.snapshotId()` is going to fail with a `NullPointerException`. So I
think the block above is intended to handle `ManifestFile` that is not adapted
from a `TrackedFile`? In that case did it come from a manifest list? I don't
know that we would ever have that case.
A case that probably will happen is getting changes from a root manifest. We
need to make sure that doesn't fail. We can assume its status is APPEND but
would need to be passed a target `snapshotId`. How does that happen? Would it
be better to make snapshot ID part of the contract in all cases rather than
trying to infer it from status?
Next, I think this also needs to handle REPLACED, which I think would need
to behave almost exactly like DELETED.
Last, MODIFIED signals that the manifest was changed in place. Either the
MDV was updated or new column files were added. In that case, I think this is
correct to target changes at the modified snapshot ID. This is also the only
time when the change bitmaps can be used, right? I think we need an additional
check once we determine the target snapshot ID to make sure the bitmaps are for
that verison.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]