amogh-jahagirdar commented on code in PR #16025: URL: https://github.com/apache/iceberg/pull/16025#discussion_r4235059891
########## format/spec.md: ########## @@ -924,26 +1043,33 @@ Fields with stats tracked in `content_stats` change based on updates like schema A simple (and recommended) way for writers to adapt existing metadata for table changes is to read manifests with the implementation's current `content_stats` type and apply schema evolution rules, such as reading `int` as `long` for promoted fields. -#### Sequence Number Inheritance +#### Inheritance + +Values for `snapshot_id`, `sequence_number`, and `file_sequence_number` are inherited from manifest metadata when `null`. That is, if the field is `null` for an entry, then the entry must inherit its value from the manifest file's metadata, stored in the snapshot root file. + +##### Sequence Number Inheritance Manifests track the sequence number when a data or delete file was added to the table. -When adding a new file, its data and file sequence numbers are set to `null` because the snapshot's sequence number is not assigned until the snapshot is successfully committed. When reading, sequence numbers are inherited by replacing `null` with the manifest's sequence number from the manifest list. +The `sequence_number` field represents the data sequence number and must never change after a file is added to the dataset. The data sequence number represents a relative age of the file content and should be used for planning which delete files apply to a data file. Review Comment: Ok, I remember what I did here now. The data sequence number changing for column updates is a few sentences below but having the must requirement upfront does read weirdly, or at least somewhat contradictory. The sentence below this line in this section: "The data sequence is inherited only if the entry is ADDED or MODIFIED by adding a column file" does cover the case. I think all we need to do is remove the "must never change" portion, and the section reads correctly and consistentlyl -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
