rdblue commented on code in PR #16425: URL: https://github.com/apache/iceberg/pull/16425#discussion_r4648772842
########## format/spec.md: ########## @@ -1748,7 +1751,7 @@ The rows in the delete file must be sorted by `file_path` then `pos` to optimize Equality delete files identify deleted rows in a collection of data files by one or more column values, and may optionally contain additional columns of the deleted row. -Equality delete files must not be added to v4 tables. Equality deletes cannot be added as an entry to a v4 manifest. Equality delete files referenced by v2 or v3 delete manifests must still be applied in a v4 table. +Equality delete files must not be added to v4 tables. Equality deletes cannot be added as an entry to a v4 manifest. Equality delete files referenced by v2 or v3 delete manifests must still be applied in a v4 table. Adding column files to a data file sets its data sequence number to that snapshot's sequence number, so existing equality delete files no longer apply to it (see [Writing Column Files](#writing-column-files)). Review Comment: Yeah, this statement is confusing and references a different (huge) section to hopefully clarify. We should fix this. First, I think this paragraph should not be repurposed. It was making a well-defined point about deprecation before. It could be improved to be slightly more clear: ```suggestion Equality delete files must not be added to v4 tables and cannot be written to new v4 leaf manifests. Equality delete files tracked by existing v2 or v3 delete manifests must still be applied in a v4 table. ``` Then we need to note that when a data sequence number changes, the writer is responsible for equality deletes: ``` When a data file is updated by adding a column file, its data sequence number is updated so that updated rows inherit a new `_last_updated_sequence_number`. Because the data sequence number changes, all equality deletes must be rewritten into the file's deletion vector. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
