aokolnychyi commented on a change in pull request #2654:
URL: https://github.com/apache/iceberg/pull/2654#discussion_r651440289
##########
File path: site/docs/spec.md
##########
@@ -254,6 +268,23 @@ Notes:
2. The width, `W`, used to truncate decimal values is applied using the scale
of the decimal column to avoid additional (and potentially conflicting)
parameters.
+#### Partition Evolution
+
+Table partitioning can be evolved by adding, removing, renaming, or reordering
partition spec fields.
+
+Changing a partition spec produces a new spec identified by a unique spec ID
that is added to the table's list of partition specs and may be set as the
table's default spec.
+
+When evolving a spec, changes should not cause partition field IDs to change
because the partition field IDs are used as the partition tuple field IDs in
manifest files.
+
+In v2, partition field IDs must be explicitly tracked for each partition
field. New IDs are assigned based on the last assigned partition ID in table
metadata.
+
+In v1, partition field IDs were not tracked, but were assigned sequentially
starting at 1000 in the reference implementation. This assignment caused
problems when reading metadata tables based on manifest files from multiple
specs because partition fields with the same ID may contain different data
types. For compatibility with old versions, the following rules are recommended
for partition evolution in v1 tables:
+
+1. Do not reorder partition fields
Review comment:
Good summary, thanks for doing this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
