ghoshp83 opened a new pull request, #4374:
URL: https://github.com/apache/iceberg-python/pull/4374

   Closes #4291.
   
   # Rationale for this change
   
   Follow-up to #4292, which handled the rewrite itself. This covers the retry 
conflict check from the issue.
   
   When a commit retries, `_validate_no_new_deletes_for_data_files` checks that 
no new delete file applies to the data files being replaced. It built its 
delete index with the operation's filter, though, so `_filter_manifest_entries` 
skipped any delete file whose column bounds can't match that filter. A partial 
delete rewrites those data files in full, so a concurrent delete outside the 
filter went unnoticed. The retry then committed the rewritten file, and the 
rows that delete removed came back.
   
   Now every delete file added since the starting snapshot is checked against 
the replaced data files. The filter is no longer used there, so `data_filter` 
and `case_sensitive` are dropped from the private function's signature. 
`_validate_no_new_delete_files`, the filter-based check, is unchanged.
   
   ## Are these changes tested?
   
   Yes. `test_partial_delete_validates_concurrent_delete_outside_the_filter` in 
`tests/table/test_commit_retry.py` runs against each catalog fixture:
   - a stale table handle starts `delete("k = 1")`;
   - meanwhile, a position delete storing the row `(2, 'b')` is committed. It 
has bounds `k = 2`, built with `commit_partition_position_delete`;
   - the retried delete must raise `ValidationException`, leaving `k = [1, 3]`.
   
   Without the fix, the test fails with `DID NOT RAISE ValidationException`, 
because the retry commits over the concurrent delete.
   
   ## Are there any user-facing changes?
   
   A partial delete, filtered overwrite or upsert that races with a delete on 
the same data file now fails validation on retry, instead of silently bringing 
the deleted rows back.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to