Hi,

I propose adding a _last_updated_time metadata column to row lineage for v4
tables. It records the timestamp of the snapshot that last updated a row,
complementing _last_updated_sequence_number with a wall-clock time.

Summary of the spec change:
* New reserved metadata column _last_updated_time (timestamptz).
* New optional tracking field snapshot_timestamp (timestamptz) in v4
manifests.
  Writers leave it null for added entries and for entries modified by
  column files. On read, a null value is inherited from the snapshot's
  timestamp-ms.
* Upgrading to v4 does not require rewriting manifest lists or manifests.
  Pre-v4 metadata is read as if snapshot_timestamp is null, so exising rows
  get the timestamp of the last snapshot before the v4 upgrade.

This works well with the earlier v4 spec change that requires monotonically
increasing snapshot timestamps (#16294). Because each snapshot's
timestamp-ms
is strictly greater than its parent's on the same branch, _last_updated_time
follows the same order as _last_updated_sequence_number.

This enables queries for *when a row was last updated* and for
*rows modified **since a given time*, without mapping sequence numbers
back to snapshots that may have expired.

*Spec PR:* https://github.com/apache/iceberg/pull/18526

Please vote in the next 72 hours:

[ ] +1 Adopt the spec change
[ ] +0 No preference
[ ] -1 Do not adopt the spec change (please explain)

Thanks, Steven

Discussion thread:
https://lists.apache.org/thread/cln7xtjy9kw2096gzqlg5c4gg8b9td10

Monotonic snapshot timestamp PR:
https://github.com/apache/iceberg/pull/16294

Reply via email to