raufer opened a new pull request, #4395: URL: https://github.com/apache/iceberg-python/pull/4395
## Rationale for this change CPython 3.15 is out (3.15.2), but PyIceberg cannot be installed on it from wheels: - `zstandard` is a core dependency, and it publishes no cp315 wheel: 0.25.0 (2025-09-14) is still its latest release. Its 3.15 PRs, indygreg/python-zstandard#357 and #372, are open. Without a C compiler, `pip install pyiceberg` fails on 3.15. - `pyiceberg` itself ships no cp315 wheel either, because `CIBW_PROJECT_REQUIRES_PYTHON` stops at `<3.15`. Python 3.14 added a zstd module to the standard library: `compression.zstd`, from [PEP 784](https://peps.python.org/pep-0784/). PyIceberg only needs one-shot compress and decompress (the Avro codec and Puffin blobs), so on 3.14+ the third-party package is not needed at all. ## What changes - `pyiceberg/table/puffin.py` uses `compression.zstd.decompress` on Python 3.14+. Below 3.14 it keeps `zstandard.ZstdDecompressor().decompress`. - `pyiceberg/avro/codecs/zstandard_codec.py` uses `compression.zstd.compress` and `decompress` on 3.14+. Below 3.14 the existing `zstandard` code is unchanged, only indented under the version check (`git diff -w` shows the real change). - In `pyproject.toml`, the dependency becomes `zstandard>=0.13.0; python_version < '3.14'`. `uv.lock` is regenerated with `uv lock`. - The wheel build adds 3.15: `CIBW_PROJECT_REQUIRES_PYTHON` becomes `>=3.11,<3.16`, and 3.15 joins the `setup-python` list. cibuildwheel v4.3.4 already builds cp315 by default. - `CIBW_TEST_SKIP: "cp315-*"` is set for now. The test step syncs the locked `dev` group, which pins pyarrow 25.0.1 and numpy 2.4.6 through `pyspark[connect]`, and neither has a cp315 wheel. Moving those pins is a separate change, and the skip can go once it lands. The version check follows the existing `sys.version_info >= (3, 14)` pattern (`uuid7` in `catalog/rest/__init__.py`), so mypy checks each branch for the interpreter it runs on. PEP 784 makes `compression.zstd` an optional module, built when libzstd is available; python-build-standalone (what uv installs) ships it on 3.14 and 3.15. A CPython built without it fails at import time, as a missing `zstandard` does today. ## Are these changes tested? The existing tests cover both paths: Puffin's `test_read_compressed_zstd`, and the `zstd`/`zstandard` manifest tests in `tests/utils/test_manifest.py`. Locally, on macOS arm64: | Python | zstandard installed | `tests/avro tests/table/test_puffin.py tests/utils/test_manifest.py tests/table/test_metadata.py` | |---|---|---| | 3.13.7 | yes (marker keeps it) | 281 passed | | 3.14.10 | no | 281 passed | | 3.15.2 | no | 281 passed | ruff (v0.14.3) and mypy (1.18.2) are clean on the changed files under 3.13 and 3.15. ## Are there any user-facing changes? On Python 3.14+, `pip install pyiceberg` no longer installs `zstandard`. On 3.15, pyiceberg installs from a wheel. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
