ShivamSharma43 opened a new issue, #2522:
URL: https://github.com/apache/iceberg-go/issues/2522
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
Deleting a column that is an identifier field succeeds and silently removes
it from the schema's identifier fields. The table loses its row identity (used
for upserts and equality deletes) without any error.
Reproduced on main (df0fe9a):
```go
sc := iceberg.NewSchemaWithIdentifiers(1, []int{1},
iceberg.NestedField{ID: 1, Name: "id", Type:
iceberg.PrimitiveTypes.Int32, Required: true},
iceberg.NestedField{ID: 2, Name: "name", Type:
iceberg.PrimitiveTypes.String},
)
md, _ := table.NewMetadata(sc, nil, table.UnsortedSortOrder, "", nil)
txn := table.New([]string{"t"}, md, "", nil, nil).NewTransaction()
newSchema, err := table.NewUpdateSchema(txn, true, true).
DeleteColumn([]string{"id"}).
Apply()
fmt.Println(err, newSchema.IdentifierFieldIDs)
```
Output:
```
<nil> []
```
Expected: an error, because `id` is an identifier field and the identifier
fields were not updated.
The current behavior is also inconsistent. These cases already fail in
`Apply` with `identifier field not found`:
- deleting the struct that contains an identifier field (for example
`person` when `person.id` is an identifier field)
- deleting the identifier column with a different case in case-insensitive
mode (`DeleteColumn([]string{"ID"})`)
**Cause:** `deleteColumn` in `table/update_schema.go` calls
`delete(u.identifierFieldNames, fullName)`. That only removes an exact name
match, which is why the cases above still fail.
**Java:** `SchemaUpdate.applyChanges` rejects a deleted identifier field, or
a deleted parent of one, with "Cannot delete identifier field ... To force
deletion, also call setIdentifierFields to update identifier fields":
https://github.com/apache/iceberg/blob/52fc764127402ade8170b1b53ddd95fde13cb75c/core/src/main/java/org/apache/iceberg/SchemaUpdate.java#L541-L562
`TestSchemaUpdate.testDeleteIdentifierFieldColumnsFails` covers this.
`testDeleteIdentifierFieldColumns` checks that deleting works when
`setIdentifierFields` is also called, in either order.
PyIceberg's `delete_column` doesn't touch the identifier field names, and
`_apply` fails with "Cannot find identifier field id. In case of deletion,
update the identifier fields first.":
https://github.com/apache/iceberg-python/blob/639a3bf505b6932945bba3f661395255bac872c2/pyiceberg/table/update/schema.py#L674-L680
**Proposed fix:** remove the `delete(u.identifierFieldNames, fullName)`
line, so the existing check in `Apply` rejects the deleted identifier field, as
PyIceberg does. `DeleteColumn` combined with `SetIdentifierField` still works
in either order.
I'd like to work on this. I have a fix and a unit test ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
