emkornfield commented on code in PR #18526: URL: https://github.com/apache/iceberg/pull/18526#discussion_r4288820068
########## format/spec.md: ########## @@ -2110,6 +2135,18 @@ Equality deletes are prohibited in v4. * Upgrading a v2 or v3 table to v4 does not require rewriting data or delete files * Readers must continue to apply equality deletes for v2 and v3 tables and for equality deletes carried over into upgraded v4 tables +Row lineage changes: + +* When writing a manifest, ADDED entries and entries MODIFIED by column files must be written with a null `snapshot_timestamp` +* When an ADDED or MODIFIED entry has a null `snapshot_timestamp`, readers must assign it the snapshot's `timestamp-ms` converted to microseconds in the root manifest, or the leaf manifest's `snapshot_timestamp` in a leaf manifest Review Comment: A weird edge-case we might want to be explicit about: Is if you rewrite a data file that was added prior to V4 it ends up with a null timestamp. If you then rewrite a second time, I think you end up populating the the timestamp field with the second rewrite because you can no longer determine the change? The alternative might would be an explicit sentinel value for unknown cases? (either something like year 1 or snapshot time at conversion). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
