rdblue commented on code in PR #16025: URL: https://github.com/apache/iceberg/pull/16025#discussion_r3938322929
########## format/spec.md: ########## @@ -656,15 +661,22 @@ A data or delete file is associated with a sort order by the sort order's id wit ### Manifests -A manifest is an immutable Avro file that lists data files or delete files, along with each file’s partition data tuple, metrics, and tracking information. One or more manifest files are used to store a [snapshot](#snapshots), which tracks all of the files in a table at some point in time. Manifests are tracked by a [manifest list](#manifest-lists) for each table snapshot. +A manifest is an immutable file that lists data files or delete files, along with each file’s partition data, metrics, and tracking information. One or more manifest files are used to store a [snapshot](#snapshots), which tracks all of the files in a table at some point in time. In v1-v3, manifests are tracked by a [manifest list](#manifest-lists) for each table snapshot. In v4, a single root manifest per snapshot can directly reference data files, delete files, and other data and delete manifests. A manifest is a valid Iceberg data file: files must use valid Iceberg formats, schemas, and column projection. A manifest may store either data files or delete files, but not both because manifests that contain delete files are scanned first during job planning. Whether a manifest is a data manifest or a delete manifest is stored in manifest metadata. -A manifest stores files for a single partition spec. When a table’s partition spec changes, old files remain in the older manifest and newer files are written to a new manifest. This is required because a manifest file’s schema is based on its partition spec (see below). The partition spec of each manifest is also used to transform predicates on the table's data rows into predicates on partition values that are used during job planning to select files from a manifest. +**Partition Spec Binding:** -A manifest file must store the partition spec and other metadata as properties in the Avro file's key-value metadata: +- v1-v3: A manifest stores files for a single partition spec. When a table’s partition spec changes, old files remain in the older manifest and newer files are written to a new manifest. This is required because a manifest file’s schema is based on its partition spec. The partition spec of each manifest is used to transform predicates on the table’s data rows into predicates on partition values during job planning. Review Comment: ```suggestion - v1-v3: A manifest stores files for a single partition spec. When a table’s partition spec changes, old files remain in the older manifest and newer files are written to a new manifest. This is required because a manifest file’s schema is based on its partition spec. ``` I would move the last sentence out to its own paragraph: > The partition spec used when writing each data file is used to transform predicates on the table’s data rows into predicates on partition values during job planning. In v3, the same partition spec is used for all data files in a manifest. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
