rdblue commented on a change in pull request #1141:
URL: https://github.com/apache/iceberg/pull/1141#discussion_r551652925
##########
File path: site/docs/spec.md
##########
@@ -646,6 +647,25 @@ Each partition field in the fields list is stored as an
object. See the table fo
In some cases partition specs are stored using only the field list instead of
the object format that includes the spec ID, like the deprecated
`partition-spec` field in table metadata. The object format should be used
unless otherwise noted in this spec.
+#### Partition Field ID Handling
+
+A partition field ID is an integer used to identify a partition field.
+These IDs should not conflict with the IDs required for the other fields in
data files.
+To avoid that, iceberg assigns partition field IDs starting at 1000.
+
+The requirements below are for different versions of tables:
+
+* For v1 tables, partition field metadata should include a field id for each
partition field, but this is not required.
+* For v2 tables, partition field metadata must include a field id for each
partition field. Partition field IDs are unique across partition specs to
support the partition spec evolution for a given table.
+
+To remove partition fields from the partition spec in an existing v1 table, it
is recommended not removing fields but replacing their transforms with
`alwaysNull`.
Review comment:
This should be in a secondary section about working with v1 tables. That
should explain the problem: that field IDs were reused, which causes problems
in metadata tables. To avoid the problem, don't reorder or delete partition
fields. Instead, add new fields at the end and replace fields with `void`. Also
note that this is not needed fro v2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
