jiangxt2 opened a new issue, #13556: URL: https://github.com/apache/gravitino/issues/13556
### What would you like to be improved? I opened [Ray Data RFC #66263](https://github.com/ray-project/ray/issues/66263) to discuss integrating Apache Gravitino as a catalog provider. In a follow-up Slack discussion, Ray Data maintainer feedback highlighted the Gravitino Python client’s exact dependency pins as a concern for a potential native provider. The published [`apache-gravitino` 1.3.0 package](https://pypi.org/pypi/apache-gravitino/1.3.0/json) pins `pyarrow==23.0.1`, `fsspec==2026.2.0`, `s3fs==2026.2.0`, and other cloud filesystem packages. Ray’s [`ray[data]` extra](https://github.com/ray-project/ray/blob/master/python/setup.py) allows `pyarrow>=17.0.0` and does not pin `fsspec`, but Ray also has a [Data CI environment fixed to PyArrow 17](https://github.com/ray-project/ray/blob/master/ci/raydepsets/configs/ci_data.depsets.yaml) and a separate [ML requirements file that pins `s3fs==2023.12.1`](https://github.com/ray-project/ray/blob/master/python/requirements/ml/core-requirements.txt). Gravitino’s pins conflict with those fixed environments, although they do not conflict with every Ray installation. This issue concerns supported version ranges for runtime dependencies. Separating base, optional storage, and development dependencies is tracked in #12211. ### How should we improve? - Establish tested compatibility ranges for runtime dependencies instead of requiring one exact version of each package. - Retain pinned development and test environments where reproducibility is needed. - Validate dependency resolution and relevant client functionality against Ray’s PyArrow 17 environment and Gravitino’s current baseline, including supported cloud filesystem combinations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
