anxkhn opened a new pull request, #3606:
URL: https://github.com/apache/iceberg-python/pull/3606
<!-- Closes #1174 -->
# Rationale for this change
PyIceberg's PyArrow UUID handling depends on the native UUID APIs
`pyarrow.uuid()`
and `pyarrow.UuidType`, but the `pyarrow`, `pandas`, `duckdb`, and `ray`
extras
still pin `pyarrow>=17.0.0`.
- On write, `PyArrowSchemaVisitor.visit_uuid` returns `pa.uuid()`
(`pyiceberg/io/pyarrow.py:806`).
- On read, `_ConvertToIceberg.primitive` branches on
`isinstance(primitive, pa.UuidType)` (`pyiceberg/io/pyarrow.py:1497`).
Both `pyarrow.uuid()` and `pyarrow.UuidType` were introduced in Apache Arrow
18.0.0 (released 2024-10-28) and do not exist in 17.x. So an environment that
resolves `pyarrow==17.x` under the current floor raises
`AttributeError: module 'pyarrow' has no attribute 'uuid'` (or `'UuidType'`)
as
soon as a UUID-typed column is written, or an Arrow schema containing a UUID
is
converted back to Iceberg.
This usage was added in #2007 ("Fix UUID support") without raising the
floor, and
the earlier `Bump PyArrow to 18.0.0` change only moved the tested/locked
version,
not the declared lower bound. This PR bumps the floor to `pyarrow>=18.0.0`
for the
four affected extras so the declared dependency matches what the code
actually
requires. See also #1174 (Minimum required pyarrow version).
## Are these changes tested?
This is a dependency-floor bump with no code change, so the real guard is CI
resolving the raised floor across the supported Python matrix; `uv lock
--check`
confirms `uv.lock` stays in sync with `pyproject.toml`.
Locally:
- `uv lock --check` passes (lock in sync).
- `make lint` (ruff, ruff-format, mypy, license, uv-lock, ...) all pass.
- Existing UUID coverage still passes and exercises both affected code paths:
`tests/io/test_pyarrow_visitor.py` (60 passed),
`tests/utils/test_schema_conversion.py -k UUID` (2 passed), and the UUID
cases in
`tests/test_conversions.py` (2 passed). A manual round-trip
`UUIDType -> pa.uuid() -> pa.UuidType -> UUIDType` runs clean.
No new test is added because there is no new code path to cover: the
existing UUID
conversion tests already exercise `visit_uuid` and `_ConvertToIceberg`, and
the
floor bump only makes the declared dependency honest. A heavier alternative
would
be to version-gate `pa.uuid()`/`pa.UuidType` at runtime (as done for
`AzureFileSystem` via `MIN_PYARROW_VERSION_SUPPORTING_AZURE_FS` in
`pyiceberg/io/pyarrow.py`), but that keeps UUID unsupported on 17.x rather
than
requiring the version the feature needs; happy to go that route instead if
preferred.
## Are there any user-facing changes?
Yes, indirectly: the minimum supported `pyarrow` for the `pyarrow`, `pandas`,
`duckdb`, and `ray` extras rises from 17.0.0 to 18.0.0. This only affects
environments currently resolving `pyarrow` 17.x, where UUID columns are
already
broken; it prevents that unsupported install rather than changing any working
behavior. (Adding the `changelog` label if this warrants a changelog entry -
I
don't have permission to set labels as an external contributor.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
