szehon-ho commented on code in PR #7581:
URL: https://github.com/apache/iceberg/pull/7581#discussion_r1207628850
##########
core/src/main/java/org/apache/iceberg/PartitionsTable.java:
##########
@@ -47,6 +47,16 @@ public class PartitionsTable extends BaseMetadataTable {
new Schema(
Types.NestedField.required(1, "partition",
Partitioning.partitionType(table)),
Types.NestedField.required(4, "spec_id", Types.IntegerType.get()),
+ Types.NestedField.required(
+ 9,
+ "last_updated_at",
+ Types.TimestampType.withZone(),
+ "Partition last updated timestamp"),
+ Types.NestedField.required(
+ 10,
+ "last_updated_snapshot_id",
Review Comment:
After some thoughts, I think both will be null if the snapshot is expired.
But I think we are no worse off than the alternative, which is to join the
entries + snapshots table for the user to find the last update time. This also
will be null if snapshot is expired.
Perhaps we can have some more persistent storage of snapshot metadata , even
after expire?
I think for this case (lastUpdateTime, lastSnapshotId), it is ok to proceed,
typically snapshots live several days at least, and I would imagine the user of
the tool is interested in the last updated partition, and can run it before the
snapshot is expired.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
