GayathriSrividya opened a new pull request, #4446:
URL: https://github.com/apache/iceberg-python/pull/4446

   Closes #4445
   
   # Rationale for this change
   
   `add_files(check_duplicate_files=True)` checked duplicate paths only before 
the first commit attempt. During a conflict retry, the snapshot producer 
rebuilt the append using refreshed metadata without repeating the 
duplicate-path validation, allowing concurrent writers to reference the same 
file twice.
   
   This change registers retry-time validation on snapshot producers and reruns 
it after refreshing metadata. Duplicate paths now raise `ValueError` before the 
retried snapshot is rebuilt.
   
   ## Are these changes tested?
   
   - `ruff check pyiceberg/table/__init__.py pyiceberg/table/update/snapshot.py 
tests/table/test_commit_retry.py` — passed
   - `ruff format --check pyiceberg/table/__init__.py 
pyiceberg/table/update/snapshot.py tests/table/test_commit_retry.py` — passed
   - `python -m compileall -q pyiceberg/table/__init__.py 
pyiceberg/table/update/snapshot.py tests/table/test_commit_retry.py` — passed
   - `git diff --check` — passed
   - The targeted pytest regression test could not complete because PyArrow 
imports stalled in the local environment.
   
   ## Are there any user-facing changes?
   
   Yes. Concurrent `add_files` calls can no longer add the same file twice 
after a commit retry. Duplicate files are rejected with the existing 
duplicate-file `ValueError`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to