rahulsmahadev opened a new pull request, #4347: URL: https://github.com/apache/iceberg-python/pull/4347
# Rationale for this change `Transaction.add_files(..., branch=...)` checks for already-referenced files only against the current snapshot (main), not against the target branch. Adding the same file to a branch twice is therefore not rejected, and reading the branch returns the file's rows twice: ```python tbl.append(main_rows) tbl.manage_snapshots().create_branch(snapshot_id=tbl.metadata.current_snapshot_id, branch_name="b").commit() tbl.add_files([path], branch="b") tbl.add_files([path], branch="b") # should raise "already referenced by table"; it commits a second copy ``` The second call succeeds, and the branch holds two live entries for `path`. The same second call on main is rejected. #3287 describes this as a pre-existing inconsistency and preserved it. This change makes the check cover the target branch's snapshot as well as the current snapshot. The existing main check stays, and a file the branch already references is now rejected too. Snapshots and manifests shared by both are read once. Related: #3287 ## Are these changes tested? Yes. The tests are in `tests/integration/test_add_files.py` and run against the memory and SQL catalogs. `test_add_files_to_branch_rejects_file_already_referenced_by_branch` and `test_add_files_to_branch_rejects_file_added_earlier_in_transaction` fail without this change (the second add is not rejected) and pass with it. `test_add_files_to_branch_accepts_distinct_file_with_same_rows` adds a different path with the same rows and passes both ways. `test_add_files_to_branch_rejects_file_referenced_only_by_main` passes both ways and pins that files referenced by main are still rejected on a branch. ## Are there any user-facing changes? Yes: `add_files` with a non-main `branch` now also rejects files that the branch already references. Files referenced by main are still rejected, and the `check_duplicate_files` docs mention the target branch. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
