jerry-024 commented on code in PR #760:
URL: https://github.com/apache/paimon-rust/pull/760#discussion_r3879746131
##########
crates/paimon/src/table/vector_search_builder.rs:
##########
@@ -1634,6 +1758,43 @@ async fn evaluate_batch_vector_search(
let index_search_limit = indexed_search_limit(max_limit, refine_factor)?;
let vector_entry_count = vector_entries.len();
+ let shared_include_row_ids =
+ vector_searches[0]
+ .effective_include_row_ids()
+ .filter(|include_row_ids| {
+ vector_searches
+ .iter()
+ .all(|search| search.effective_include_row_ids() ==
Some(*include_row_ids))
Review Comment:
<!-- dlf-review -->
**[MAJOR]** Detecting the shared pre-filter compares the entire
`RoaringTreemap` once for every query in the batch, even though
`BatchVectorSearchBuilder` installs the same `Arc` on every query.
Inspect `shared_include_row_ids` directly and use `Arc::ptr_eq` against the
first query Arc; fall back to per-query localization if any query does not
share that `Arc`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
