zeroshade commented on code in PR #135:
URL:
https://github.com/apache/terraform-provider-iceberg/pull/135#discussion_r4620233011
##########
internal/provider/table_schema.go:
##########
@@ -321,6 +323,44 @@ func validateSchemaEvolution(current, planned
*iceberg.Schema) error {
return nil
}
+// validateNewFieldIDs rejects a planned field whose id is not in the current
+// schema but was already assigned, by a past schema or up to last-column-id.
+func validateNewFieldIDs(metadata table.Metadata, planned *iceberg.Schema)
error {
+ currentByID, err := iceberg.IndexByID(metadata.CurrentSchema())
+ if err != nil {
+ return err
+ }
+ plannedNames, err := iceberg.IndexNameByID(planned)
+ if err != nil {
+ return err
+ }
+
+ pastNames := map[int]string{}
Review Comment:
Nit, non-blocking: every id in a past schema is also `<= last-column-id`, so
this lookup only improves the error message. Worth keeping for that; maybe say
so in a comment.
##########
internal/provider/table_schema.go:
##########
@@ -321,6 +323,44 @@ func validateSchemaEvolution(current, planned
*iceberg.Schema) error {
return nil
}
+// validateNewFieldIDs rejects a planned field whose id is not in the current
+// schema but was already assigned, by a past schema or up to last-column-id.
+func validateNewFieldIDs(metadata table.Metadata, planned *iceberg.Schema)
error {
+ currentByID, err := iceberg.IndexByID(metadata.CurrentSchema())
+ if err != nil {
+ return err
+ }
+ plannedNames, err := iceberg.IndexNameByID(planned)
+ if err != nil {
+ return err
+ }
+
+ pastNames := map[int]string{}
+ for _, s := range metadata.Schemas() {
+ names, err := iceberg.IndexNameByID(s)
+ if err != nil {
+ return err
+ }
+ maps.Copy(pastNames, names)
+ }
+
+ lastColumnID := metadata.LastColumnID()
+ for _, id := range slices.Sorted(maps.Keys(plannedNames)) {
+ if _, ok := currentByID[id]; ok {
+ continue
+ }
+ name := plannedNames[id]
+ if past, ok := pastNames[id]; ok {
+ return fmt.Errorf("field %q has id %d, which belonged
to dropped field %q; ids of dropped fields cannot be reused", name, id, past)
+ }
+ if id <= lastColumnID {
Review Comment:
On "stricter than the spec" in the description: I don't think it is. The
spec describes `last-column-id` as ensuring "columns are always assigned an
unused ID", and writers may allocate ids from it for stats structs that never
appear in a schema. So this branch prevents real collisions, and I'd lean
against later relaxing it for same-type reuse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]