weiqingy opened a new pull request, #1001:
URL: https://github.com/apache/flink-agents/pull/1001
Linked issue: #999
### Purpose of change
`queryEmbedding` attached its filter with `post_filter`, which Elasticsearch
applies after the KNN phase has already selected its `k` nearest hits. The
filter could then only discard documents from that set, and never reach
matching documents that fell outside the top `k`. A caller asking for five
documents matching `user_id=alice` could get none back while hundreds were
indexed, whenever the nearest vectors belonged to other users.
This moves the filter into the KNN clause, so the nearest neighbors are
selected from among the documents that match. `get` and `delete` already build
the same filter map into a real query clause, so this also removes the
divergence between the three methods.
Both filter forms are affected, since the unified `filters` map and a raw
`filter_query` are merged into a single query by `combineQueryJson` before it
is attached.
### Tests
`ElasticsearchVectorStoreTest#testQueryEmbeddingFiltersBeforeSelectingNeighbors`
indexes six documents pointing along the query vector and three orthogonal
documents belonging to another user, then queries with `k` of 5 and a filter
selecting the three. It fails with `expected: <3> but was: <0>` before the
change and passes after, run against Elasticsearch 8.19.0.
The test uses `addEmbedding` with explicit vectors so the result depends
only on where the filter is applied.
Note that `ElasticsearchVectorStoreTest` is annotated `@Disabled("Should
setup Elasticsearch server.")`, so this test does not run in CI and needs a
local Elasticsearch, as with the rest of the class.
### API
No signature changes. This does change observable behavior: callers passing
a filter to `queryEmbedding` previously received at most the matching subset of
the top `k`, and now receive up to `k` matching documents.
### Documentation
- [ ] `doc-needed`
- [ ] `doc-not-needed`
- [x] `doc-included`
### Was this patch authored or co-authored using generative AI tooling?
- [x] Yes
- [ ] No
Generated-by: Claude Code 2.1.228 (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
