FANNG1 opened a new issue, #66703: URL: https://github.com/apache/doris/issues/66703
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version `branch-4.1` and `master` (both pin `lance-c` v0.1.2 in `thirdparty/vars.sh`, which builds against lance-rs 4.0.1). ### What's Wrong? `vector_search()` over a `FixedSizeList<Float16>` or `FixedSizeList<Float64>` column **that has an IVF vector index** panics inside the Lance Rust code, and because the FFI boundary has no `catch_unwind`, the panic aborts the whole BE process. ``` thread 'lance-cpu' panicked at arrow-array-57.3.0/src/cast.rs:840:33: primitive array *** SIGABRT ... received by PID <be pid> ``` The client only sees the connection die: ``` ERROR 1105 (HY000): RpcException, msg: send fragments failed. io.grpc.StatusRuntimeException: UNAVAILABLE: io exception ``` `cast.rs:840` is the `as_primitive` downcast helper, so the index search path appears to assume Float32 vectors. Scope, verified by isolating each case with a freshly restarted BE and counting new panics in `be.out`: | Vector column type | flat search (`use_index=false`) | **indexed search** | |---|---|---| | Float32 | ok | ok | | **Float16** | ok | **BE aborts** | | **Float64** | ok | **BE aborts** | | UInt8 (hamming index) | ok | ok | So the trigger is specifically *indexed* search on a non-Float32 float column. Flat search over the same columns is fine, which is why the existing `all_types.lance` fixture — which has five vector columns but no index at all — never surfaced it. ### What You Expected? A user query must never abort the backend. Either the search works, or it returns an error. ### How to Reproduce? The repo has no fixture combining a non-Float32 vector column with an index, so the table has to be created first (pylance, against the MinIO started by `docker/thirdparties/run-thirdparties-docker.sh -c iceberg`): ```python import io, lance, lance_namespace, pyarrow as pa, pyarrow.ipc as ipc from lance_namespace_urllib3_client.models import CreateTableRequest ns = lance_namespace.connect("dir", { "root": "s3://warehouse/lance", "storage.endpoint": "http://127.0.0.1:19001";, "storage.access_key_id": "admin", "storage.secret_access_key": "password", "storage.region": "us-east-1", "storage.aws_allow_http": "true"}) so = {"endpoint": "http://127.0.0.1:19001";, "access_key_id": "admin", "secret_access_key": "password", "region": "us-east-1", "allow_http": "true"} DIM, ROWS = 16, 1024 emb = pa.FixedSizeListArray.from_arrays( pa.array([float(r + j) for r in range(ROWS) for j in range(DIM)], type=pa.float16()), DIM) t = pa.table({"row_id": pa.array(range(1, ROWS + 1), type=pa.int64()), "embedding": emb}) buf = io.BytesIO() with ipc.new_stream(buf, t.schema) as w: w.write_table(t) loc = ns.create_table(CreateTableRequest(id=["doris", "f16_repro"]), buf.getvalue()).location lance.dataset(loc, storage_options=so).create_index( "embedding", "IVF_FLAT", name="idx", metric="L2", num_partitions=4) ``` Then, from Doris: ```sql CREATE CATALOG lance_repro PROPERTIES ( "type" = "lance", "lance.catalog.type" = "filesystem", "warehouse" = "s3://warehouse/lance", "s3.endpoint" = "http://127.0.0.1:19001";, "s3.access_key" = "admin", "s3.secret_key" = "password", "s3.region" = "us-east-1", "use_path_style" = "true"); -- aborts the BE SELECT row_id FROM vector_search( "table" = "lance_repro.doris.f16_repro", "column" = "embedding", "query_vector" = "[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]", "top_k" = "3", "nprobes" = "4"); -- same table, same query, no index: works SELECT row_id FROM vector_search( "table" = "lance_repro.doris.f16_repro", "column" = "embedding", "query_vector" = "[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]", "top_k" = "3", "use_index" = "false"); ``` Swap `pa.float16()` for `pa.float64()` to reproduce the Float64 case. Note when reading `be.out`: it is appended across restarts, so compare the count of `panicked at` lines before and after the query rather than just grepping for the message. ### Analysis Not a Doris misuse of the C ABI. Running the same query against the same tables through pylance 7.0.0 works for Float16, Float64 and Float32 alike, and it accepts a query vector of any float width against a Float16 column — so passing the column's native element type, which is what `lance_reader.cpp` and `lance_scanner_nearest` do, is correct usage. The fault is in the pinned lance-rs 4.0.1. Rebuilding with **lance-c v0.1.6** (which moves the lance dependency from crates.io `4.0.1` to a newer git rev) makes both cases pass, verified locally: | | lance-c 0.1.2 | lance-c 0.1.6 | |---|---|---| | Float16 + IVF_FLAT | BE aborts | returns `1, 2, 3` | | Float64 + IVF_FLAT | BE aborts | returns `1, 2, 3` | #66698 is already upgrading lance-c to v0.1.6 on `master`. One extra note for whoever lands it: on macOS the newer dependency tree needs `-framework IOKit` added to the BE link libraries, otherwise `doris_be` fails to link with undefined `IORegistryEntry*` / `IOService*` symbols. Two things worth treating separately from the version bump: 1. **The FFI boundary has no panic guard.** Upgrading fixes these two cases, but any `panic!` anywhere in lance-rs still takes the BE down. A `catch_unwind` around the lance-c calls, translating a panic into a `Status`, would contain the whole class. Two other reachable panics found the same way: `vector_search` on an `Int8` column with NULLs (`fixed_size_list_array.rs:142`, still reproducible on v0.1.6), and building a hamming index on a float column (`kmeans.rs:382`). 2. **Test coverage has a hole shaped exactly like this bug.** Every indexed fixture is Float32, and the only multi-type fixture (`all_types.lance`) has no index, so "non-Float32 column *with* an index" was never exercised. #66512 adds fixtures for those cells plus a suite that queries them. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
