stevenzwu commented on code in PR #19019:
URL: https://github.com/apache/iceberg/pull/19019#discussion_r4609683581


##########
core/src/main/java/org/apache/iceberg/TrackingStruct.java:
##########
@@ -62,6 +63,7 @@ class TrackingStruct extends SupportsIndexProjection 
implements Tracking, Serial
   private Long firstRowId = null;
   private byte[] deletedPositions = null;
   private byte[] replacedPositions = null;
+  private Long snapshotTimestampMicros = null;

Review Comment:
   Note that `inherit(long, long)` below doesn't fill `snapshotTimestampMicros` 
yet. That's intentional: this PR is limited to the schema, interface, and 
struct changes. 
   
   Read inheritance comes in the next stacked PR. There, `inherit` takes the 
manifest's snapshot timestamp as a third argument and fills it under the same 
condition as the data sequence number, and the root manifest writer persists 
the commit timestamp for added and modified entries.
   



##########
core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java:
##########
@@ -341,19 +347,31 @@ public void statusFilter(FileFormat format) throws 
IOException {
             // inherits seq number and assigned first row ID
             dataFile(
                 new TrackingStruct(
-                    EntryStatus.ADDED, 42L, MANIFEST_SEQ, MANIFEST_SEQ, null, 
10_000L, null, null),
+                    EntryStatus.ADDED,
+                    42L, // snapshotId
+                    MANIFEST_SEQ,
+                    MANIFEST_SEQ,
+                    null, // modifiedSnapshotId
+                    10_000L, // firstRowId
+                    null, // deletedPositions
+                    null, // replacedPositions
+                    null), // snapshotTimestampMicros

Review Comment:
   `null` because timestamp inheritance on read isn't in this PR (the sequence 
number and first row ID here are inherited). The next stacked PR changes this 
to the manifest's timestamp. Same for the added entry in `mdvFilter`.



##########
core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java:
##########
@@ -585,12 +635,20 @@ public void inheritanceAddedSequenceNumbers(FileFormat 
format) throws IOExceptio
             // inherits sequence numbers
             dataFile(
                 new TrackingStruct(
-                    EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, null, 5_000L, 
null, null),
+                    EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, null, 5_000L, 
null, null, null),

Review Comment:
   `null` because timestamp inheritance on read isn't in this PR. The next 
stacked PR changes this to the manifest's timestamp.



##########
core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java:
##########
@@ -516,11 +557,12 @@ public void 
inheritanceModifiedSnapshotIdNotInherited(FileFormat format) throws
     TrackedFile withModifiedSnapshotId =
         dataFile(
             new TrackingStruct(
-                EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, 1234567L, 5_000L, 
null, null),
+                EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, 1234567L, 5_000L, 
null, null, SNAPSHOT_TS),

Review Comment:
   Unrelated to this PR, but noticed while filling in timestamps: 
`withModifiedSnapshotId` here is an ADDED entry with a non-null modified 
snapshot ID. Since #18984, `TrackingBuilder.dvUpdated()` no longer sets a 
modified snapshot ID on added entries, so this fixture describes a state the 
builder can't produce. The test still passes because it only checks that the 
modified snapshot ID isn't inherited. A carried-forward EXISTING entry that was 
modified earlier would be a realistic replacement. I left it alone to keep this 
PR focused, and can follow up separately.
   
   



##########
core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java:
##########
@@ -639,11 +697,13 @@ public void 
inheritanceModifiedDataSequenceNumber(FileFormat format) throws IOEx
   public void inheritanceFirstRowId(FileFormat format) throws IOException {
     TrackedFile withoutFirstRowId =
         dataFile(
-            new TrackingStruct(EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, null, 
null, null, null),
+            new TrackingStruct(
+                EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, null, null, null, 
null, SNAPSHOT_TS),

Review Comment:
   Note that `withoutFirstRowId` isn't a realistic production state: an added 
entry wouldn't have a explicit file sequence number and a snapshot timestamp 
without a first row ID. In practice the file sequence number, first row ID, and 
snapshot timestamp are either all assigned by the root writer at commit time or 
all inherited together on read. The fixture keeps the other fields explicit on 
purpose, so first row ID is the only value this test exercises.
   



##########
core/src/main/java/org/apache/iceberg/V4Metadata.java:
##########
@@ -993,28 +995,20 @@ public <T> void set(int pos, T value) {
 
     @Override
     public <T> T get(int pos, Class<T> javaClass) {
-      return javaClass.cast(get(pos));
+      return javaClass.cast(getInternal(pos));
     }
 
-    private Object get(int pos) {

Review Comment:
   The tracking wrapper is split into leaf and root `getInternal` 
implementations because the two write schemas no longer share a prefix: leaf 
tracking omits `deleted_positions` and `replaced_positions`, so the appended 
`snapshot_timestamp` is at position 6 in leaf manifests and 8 in root 
manifests. 
   
   The shared switch only worked before because the leaf tracking schema 
happened to be a prefix of the root schema. One switch per schema doesn't rely 
on that, which should be more robust as fields are added, and it follows the 
same pattern as `LeafTrackedFileWrapper` and `RootTrackedFileWrapper`.
   



##########
core/src/test/java/org/apache/iceberg/TestV4ManifestWriter.java:
##########
@@ -272,7 +281,8 @@ public void rootDataFileAppend() throws IOException {
                   null, // not modified
                   COMMIT_FIRST_ROW_ID,
                   null, // no change bitmaps
-                  null));
+                  null,
+                  null)); // snapshot timestamp

Review Comment:
   `null` even though `FILE_A` has a timestamp: `append` starts fresh from 
`TrackingBuilder.added`, and the root writer doesn't write the commit timestamp 
in this PR. The next stacked PR changes this to `COMMIT_SNAPSHOT_TS`. Same for 
the other append tests in this file.
   



##########
core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java:
##########
@@ -625,7 +683,7 @@ public void 
inheritanceModifiedDataSequenceNumber(FileFormat format) throws IOEx
         List.of(
             dataFileWithColumnFiles(
                 new TrackingStruct(
-                    EntryStatus.MODIFIED, SNAPSHOT_ID, 5L, 3L, 34L, 5_000L, 
null, null),
+                    EntryStatus.MODIFIED, SNAPSHOT_ID, 5L, 3L, 34L, 5_000L, 
null, null, null),

Review Comment:
   `null` because timestamp inheritance on read isn't in this PR. Like the data 
sequence number, the timestamp is reset when column files are updated, so the 
next stacked PR changes this to the manifest's timestamp.
   
   



##########
core/src/test/java/org/apache/iceberg/TestV4ManifestReader.java:
##########
@@ -706,23 +776,31 @@ public void 
inheritanceFirstRowIdIncrementsByRecordCount(FileFormat format) thro
         List.of(
             dataFile(
                 new TrackingStruct(
-                    EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, null, 10_000L, 
null, null),
+                    EntryStatus.ADDED, SNAPSHOT_ID, 5L, 5L, null, 10_000L, 
null, null, null),

Review Comment:
   `null` because timestamp inheritance on read isn't in this PR. The next 
stacked PR changes these to the manifest's timestamp.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to