jiangxt2 opened a new pull request, #13723: URL: https://github.com/apache/gravitino/pull/13723
### What changes were proposed in this pull request? Add a Gravitino-owned Daft Native contract test against a real Gravitino Iceberg REST service, using the dynamic configuration provider, a memory-backed catalog, and a temporary local warehouse. The scope and fixture lifecycle follow the Ray Data contract-test work in [#13325](https://github.com/apache/gravitino/pull/13325). - Verify empty-table schema and reads, two append batches including NULL values, field order, complete row values, filtering, and independent PyIceberg readback after reloading the table through REST. - Add an owned-server fixture with configuration restoration and resource cleanup, plus unit tests covering partial setup, HTTP response failures, cleanup errors, and required dependencies. - Pin optional Daft/PyIceberg dependencies in an isolated environment and add dedicated `daftIcebergDependencies` and `daftIcebergIT` Gradle tasks. - Add a dedicated Python 3.12/JDK 17 CI job with retained logs and reproduction instructions in `docs/iceberg-rest-engine/daft.md`. ### Why are the changes needed? Gravitino needs regression coverage for Daft reading and appending to existing Iceberg tables through its REST service. This test detects failures in service configuration, catalog loading, and table commits within Gravitino's test suite. Dependency isolation is necessary because the Gravitino Python client pins `cachetools==7.1.4`, while the PyIceberg version supported by this Daft baseline requires `cachetools<7`. The fixture creates Gravitino metadata through HTTP, and the dedicated test environment installs only its own requirements. Fix: #13720 ### Does this PR introduce _any_ user-facing change? Adds Daft integration documentation and a contributor test task. No production APIs or configuration keys change. The contract covers Daft Native reads and append writes for existing tables; Daft Catalog discovery, GVFS, credential vending, and distributed execution remain outside its scope. ### How was this patch tested? Local validation used macOS arm64, Corretto JDK 17.0.18, Gradle 8.2, and Python 3.12.14 with Daft 0.7.26, PyIceberg 0.11.1, PyArrow 23.0.1, and requests 2.32.5. - Dedicated `daftIcebergIT`: 18 tests passed (17 fixture unit tests and one real Daft contract test), with zero skips. The original configuration was restored and the owned server process exited. - Fixture unit tests: 17 tests passed on each of Python 3.10.21 and Python 3.12.14, with zero skips. - Distribution compilation, dependency installation and `pip check`, Ruff, Black, Pylint, Actionlint, RAT, and diff whitespace checks passed. To reproduce from the repository root with JDK 17: ```bash ./gradlew compileDistribution -PskipWeb=true -x test ./gradlew -PpythonVersion=3.12 -PskipDockerTests=true -PskipWeb=true \ :clients:client-python:daftIcebergIT ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
