rahulsmahadev opened a new pull request, #4345:
URL: https://github.com/apache/iceberg-python/pull/4345
# Rationale for this change
`delete()` and `overwrite()` accept `case_sensitive=False`, and the
operation binds its filter that way. When a concurrent commit lands first, the
retry validates the filter against the new snapshots, but the helpers in
`pyiceberg/table/update/validate.py` bind it with the default
`case_sensitive=True`. A filter that matches the column only case-insensitively
then fails, even when the concurrent commit does not conflict:
```python
tbl1 = catalog.load_table("default.t") # column "Id"
tbl2 = catalog.load_table("default.t")
tbl1.append(pa.table({"Id": [0]}))
tbl2.delete("id > 5", case_sensitive=False)
# ValueError: Could not find field with name id, case_sensitive=True
```
The same delete written with the exact column case commits. This change
passes the operation's `case_sensitive` setting through the conflict-detection
helpers to the metrics evaluator they use, as Java's `MergingSnapshotProducer`
does with its `caseSensitive` flag. The new parameter defaults to `True`, so
other callers are unchanged.
## Are these changes tested?
Yes. Two tests in `tests/table/test_commit_retry.py`, run against each
catalog of the `catalog` fixture, both fail without the fix:
- `test_concurrent_append_case_insensitive_delete_retries_successfully`:
after a non-conflicting concurrent append, the case-insensitive delete retries
and commits.
-
`test_concurrent_append_case_insensitive_delete_raises_validation_exception`: a
conflicting concurrent append still raises `ValidationException`, not
`ValueError`.
## Are there any user-facing changes?
Bug fix only: case-insensitive deletes and overwrites now retry after a
concurrent commit instead of raising `ValueError`. No API changes.
This pull request and its description were written by Isaac.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
