qzyu999 opened a new pull request, #3813: URL: https://github.com/apache/iceberg-python/pull/3813
## Summary Move PyArrow-specific table operations (joins, group_by, duplicate detection, row comparison) from `table/upsert_util.py` into `pyiceberg/io/pyarrow.py`. The `upsert_util` module now delegates to helpers in `io/pyarrow` rather than importing `pyarrow` directly. ## Changes - New functions in `pyiceberg/io/pyarrow.py`: `_upsert_unique_keys`, `_upsert_has_duplicate_rows`, `_upsert_get_rows_to_update` - Modified `pyiceberg/table/upsert_util.py`: removed direct `import pyarrow` / `from pyarrow import compute`, now imports helpers from `pyiceberg.io.pyarrow` ## What stays the same - All existing `from pyiceberg.table.upsert_util import ...` imports continue to work - Zero behavior change, pure code movement - No new dependencies ## Why This consolidates PyArrow logic behind the `io/pyarrow` module boundary. `table/upsert_util.py` was the only file in `pyiceberg/table/` with a top-level `import pyarrow` and inline PyArrow compute operations (joins, aggregations, etc.). Moving these operations into `io/pyarrow.py` means all runtime PyArrow usage routes through one module, which is a precondition for the decomposition proposed in #3737. See #3812 for the full audit and tracking. ## Related - #3812 - Consolidate PyArrow logic before decomposition (tracking issue) - #3737 - Decompose io/pyarrow.py into focused modules - #3738 - Extract PyArrowFileIO (first decomposition step) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
