amogh-jahagirdar commented on code in PR #16025: URL: https://github.com/apache/iceberg/pull/16025#discussion_r4234999396
########## format/spec.md: ########## @@ -924,26 +1043,33 @@ Fields with stats tracked in `content_stats` change based on updates like schema A simple (and recommended) way for writers to adapt existing metadata for table changes is to read manifests with the implementation's current `content_stats` type and apply schema evolution rules, such as reading `int` as `long` for promoted fields. -#### Sequence Number Inheritance +#### Inheritance + +Values for `snapshot_id`, `sequence_number`, and `file_sequence_number` are inherited from manifest metadata when `null`. That is, if the field is `null` for an entry, then the entry must inherit its value from the manifest file's metadata, stored in the snapshot root file. + +##### Sequence Number Inheritance Manifests track the sequence number when a data or delete file was added to the table. -When adding a new file, its data and file sequence numbers are set to `null` because the snapshot's sequence number is not assigned until the snapshot is successfully committed. When reading, sequence numbers are inherited by replacing `null` with the manifest's sequence number from the manifest list. +The `sequence_number` field represents the data sequence number and must never change after a file is added to the dataset. The data sequence number represents a relative age of the file content and should be used for planning which delete files apply to a data file. Review Comment: Ugh good catch. I fixed this earlier but when I shuffled things later, this older requirement came back. It's true for v1-v3, but not v4. let me fix it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
