maomaodev opened a new issue, #14200:
URL: https://github.com/apache/gravitino/issues/14200
### What would you like to be improved?
### Environment
- Spark 3.5.8 with the lance-spark connector built from main (includes
lance-format/lance-spark#860) and Lance 12.0.0
- Catalog configured as `spark.sql.catalog.lance_catalog.impl=rest` pointing
at the Gravitino lance-rest-server
- Verified against Gravitino 1.3.0-era lance-rest-server behavior (same
deployment class as #13065)
### Description
This is a follow-up to #13065, which reported that native vector search was
unavailable through the Gravitino Lance REST namespace and was resolved
connector-side by lance-format/lance-spark#860 (the distributed execution path
does not call `queryTable`).
After that fix, the **remaining native search surfaces are still
unavailable** on the same deployment:
1. **Full-text search** — the `lance_match` / `lance_match_phrase` /
`lance_multi_match` predicates and the `SEARCH` table function
2. **`HYBRID_SEARCH`** — hybrid vector + full-text retrieval
Both dispatch their queries to `POST /v1/table/{id}/query`, which the
lance-rest-server does not implement, and fail with:
```
org.apache.spark.SparkException: Job aborted due to stage failure ...
InternalException{code=18, message='Internal error: Failed to parse error
response:
status=404 Not Found, body=<html>...'}
at org.lance.namespace.RestNamespace.queryTable(RestNamespace.java:377)
at
org.lance.spark.search.LanceSearchColumnarPartitionReader.openArrowReader(...)
```
Everything else works through the same namespace: table resolution, DDL/DML,
and `CREATE`/`SHOW`/`DROP INDEX` (which the connector executes against the
dataset directly, with storage options vended via `describeTable`). Only query
execution is affected — same boundary as #13065.
### Reproduction
With a REST catalog backed by the Gravitino lance-rest-server (namespace
options `impl=rest` plus the server URI, and the connector's Spark SQL
extensions enabled):
```sql
CREATE TABLE lance_catalog.default.docs (id INT, body STRING, vector
ARRAY<FLOAT>)
USING lance TBLPROPERTIES ('vector.arrow.fixed-size-list.size'='2');
INSERT INTO lance_catalog.default.docs VALUES
(1, 'spark sql', array(0.1, 0.2)),
(2, 'other text', array(0.8, 0.9));
ALTER TABLE lance_catalog.default.docs CREATE INDEX body_fts USING fts
(body);
-- FTS predicate: fails with the 404 above
SELECT * FROM lance_catalog.default.docs WHERE lance_match(body, 'spark');
-- SEARCH table function: fails with the 404 above
SELECT id, _score FROM SEARCH(
table => 'lance_catalog.default.docs',
query => 'spark',
search_columns => array('body')
) ORDER BY _score DESC;
-- HYBRID_SEARCH: fails with the 404 above
SELECT id, _relevance_score FROM HYBRID_SEARCH(
table => 'lance_catalog.default.docs',
query_vector => array(0.1, 0.2),
query => 'spark',
vector_column => 'vector',
search_columns => array('body')
) ORDER BY _relevance_score DESC;
-- For comparison, VECTOR_SEARCH succeeds on the same deployment when
-- spark.sql.lance.search.distributed.enabled=true (#860):
SELECT id, _distance FROM VECTOR_SEARCH(
table => 'lance_catalog.default.docs',
query_vector => array(0.1, 0.2),
vector_column => 'vector',
num_results => 2
) ORDER BY _distance;
```
### Impact
`spark.sql.lance.search.distributed.enabled=true`, the #860 workaround, only
covers `VECTOR_SEARCH` and has no effect on these surfaces (the connector docs
state this explicitly for `HYBRID_SEARCH`). As a result, Spark users on
Gravitino-managed Lance tables cannot combine Gravitino's unified
metadata/authorization with Lance native full-text or hybrid retrieval — two of
the three native search surfaces remain blocked.
### Analysis (connector side, for context)
- **FTS is a routing gap, not a missing feature
(https://github.com/lance-format/lance-spark/pull/501).** The lance-spark
connector has a complete local per-fragment FTS scan path (used by catalog-only
namespaces such as Glue/Hive), but its dispatch decision
(`supportsQueryTable`) detects a client-side method override rather than server
capability, so `impl=rest` always routes server-side and never falls back
locally.
- **`HYBRID_SEARCH` has no local path** in the connector; both side queries
are hardcoded to `queryTable`.
### Ask
1. If the resolution is expected to stay connector-side (as with #13065), we
will file the lance-spark issues/PRs and link them here.
2. Independently, and as also suggested by @FANNG1 in #13065: for namespace
routes that are part of the Lance namespace spec but intentionally not
implemented (`/query`, `/create_index`, `/index/list`, `/index/{name}/drop`),
returning a structured "operation not supported" error instead of the
container's default 404 HTML page would make client-side failures diagnosable
and distinguishable from a wrong endpoint.
### How should we improve?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
