FANNG1 commented on PR #11060: URL: https://github.com/apache/gravitino/pull/11060#issuecomment-4505305979
## The lance-ray compatibility rationale in the docs is incorrect I empirically tested the lance-ray version compatibility claim in `docs/lance-rest-integration.md`. The conclusion (0.3.0/0.4.2 work, <=0.2.0 don't) holds for a default `pip install`, but **the stated reason is wrong, and "0.2.0 not supported" is a dependency-resolution artifact rather than a real incompatibility.** ### What the doc says > lance-ray 0.2.0 and earlier are *not* supported on 1.3.0 because pip resolves them with an older `lance-namespace` whose request schema is incompatible with the upgraded server-side `lance-namespace-core` (0.7.5+). ### Test 1 — dependency resolution (`pip install --dry-run`, fresh resolve) Every lance-ray version resolves to the **same, newest** stack: | lance-ray | resolved pylance | resolved lance-namespace | |---|---|---| | 0.0.6 / 0.0.8 / 0.1.0 / 0.2.0 / 0.3.0 / 0.4.2 | 6.0.1 | **0.7.7** | pip never resolves an *older* lance-namespace — all the constraints are lower bounds (`>=`), so pip always picks the newest. The premise "pip resolves them with an older lance-namespace" is factually wrong. ### Test 2 — functional `write_lance` + `read_lance` (local `dir` namespace) | lance-ray | with pylance 6.0.1 (pip default) | |---|---| | 0.2.0 | FAIL — `ImportError: cannot import name 'LanceNamespaceStorageOptionsProvider' from 'lance'` | | 0.3.0 | OK | | 0.4.2 | OK | The break has nothing to do with the lance-namespace wire schema. lance-ray 0.2.0's code imports `LanceNamespaceStorageOptionsProvider` from `lance` (pylance), and that symbol was **removed in pylance 6.x**. ### Test 3 — lance-ray 0.2.0 with a matching pylance `LanceNamespaceStorageOptionsProvider` is present in pylance 3.x/4.x, absent in 6.x. So: ``` lance-ray 0.2.0 + pylance 4.0.1 + lance-namespace 0.7.7 -> WRITE+READ OK (100 rows) ``` **lance-ray 0.2.0 is not inherently incompatible** — it works once pylance is pinned to 3.x/4.x. ### Root cause This is a 3-way version problem (`lance-ray` <-> `pylance` <-> `lance-namespace`), not the 2-way matrix the doc presents. lance-ray 0.2.0 declares `pylance>=2.0.0` with **no upper bound**, so a default `pip install` drags in pylance 6.x and breaks it. `requirements-lance.txt` pins `lance-ray` and `ray` but **not `pylance`**, so the resolution is left to drift — that is what the test environment actually exercises. ### Suggestions 1. Pin `pylance` explicitly in `requirements-lance.txt` alongside `lance-ray`, otherwise the resolved stack is non-deterministic. 2. Rewrite the compatibility note: the constraint comes from `lance-ray` <-> `pylance` API compatibility, not from a lance-namespace schema mismatch. Ideally document the compatible `pylance` range per `lance-ray` version. 3. Separately, note the real schema risk in the **opposite** direction: pip installs the lance-namespace **0.7.7** client, while this PR's server uses lance-namespace-core **0.7.5** — the client is newer than the server. Consider bumping the server to 0.7.7 to match. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
