FANNG1 opened a new issue, #67026:
URL: https://github.com/apache/doris/issues/67026
### Search before asking
- [X] I had searched in the issues and found no similar issues.
### Version
`branch-4.1` at `5dcf1066af4` ([opt](lance) Support distributed execution
for vector_search #66841)
### What's Wrong?
`test_lance_vector_search` fails at the `boundary_single_probe` check on
`branch-4.1`. The `nprobes=1` IVF_PQ query returns a different candidate set
than the golden file: only neighbours **above** the query point come back, and
every neighbour below it is missing.
```sql
SELECT row_id, label, _distance
FROM vector_search(
"table"="test_lance_vector_search.doris.vs_ivf_pq_f32",
"column"="embedding",
"query_vector"="[255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270]",
"top_k"="9", "metric"="l2", "nprobes"="1",
"refine_factor"="10", "use_index"="true")
ORDER BY _distance, row_id
```
Expected
(`regression-test/data/external_table_p0/lance/test_lance_vector_search.out`):
```
256 0.0
255 16.0
257 16.0
254 64.0 <- missing
258 64.0
253 144.0 <- missing
259 144.0
252 256.0 <- missing
251 400.0 <- missing
```
Actual:
```
256 0.0
255 16.0
257 16.0
258 64.0
259 144.0
260 256.0
261 400.0
262 576.0
263 784.0
```
The framework reports it as a cell mismatch on line 4 (`Expect cell is: 254
/ But real is: 258`), which reads like a tie-break ordering problem, but it is
not: the returned rows themselves differ. The result set is now strictly
`row_id >= 256`, i.e. one side of the query point only.
The failure is **deterministic** — reproduced on three consecutive runs,
byte-identical each time.
### What I verified
- Reproduced on a clean build of `5dcf1066af4` with no other patches applied.
- `git diff 5dcf1066af4^ 5dcf1066af4 --
regression-test/data/external_table_p0/lance/test_lance_vector_search.out` does
**not** touch the `boundary_single_probe` block; #66841 updated only the
`post_search_filter` blocks in that file.
- The other six suites under
`regression-test/suites/external_table_p0/lance/` all pass on the same build,
including the new `test_lance_vector_search_two_phase`.
### What I did not verify
I did not build `5dcf1066af4^` to confirm the case passed before that
commit, so I cannot say from my own testing whether #66841 introduced this or
whether it was already failing. What is established is that the expectation
predates #66841 and no longer holds after it.
The `boundary_single_probe` case exists specifically to prove the IVF index
is used rather than silently falling back to a flat scan (the assertion just
above it compares the `nprobes=1` distances against the flat search), so a
one-sided candidate set here is worth a look rather than a golden refresh.
### What You Expected?
`nprobes=1` returns the candidate set the golden file records, with
neighbours on both sides of the query point.
### How to Reproduce?
```
sh run-regression-test.sh --run -s test_lance_vector_search
```
against the docker iceberg/MinIO environment
(`docker/thirdparties/run-thirdparties-docker.sh -c iceberg`), with
`enableIcebergTest = true`.
### Anything Else?
Found while merging `branch-4.1` into #66805, which touches the same Lance
files. Filing separately because it reproduces without that PR.
cc @zhangstar333
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
