JingsongLi commented on code in PR #760:
URL: https://github.com/apache/paimon-rust/pull/760#discussion_r3900608569
##########
crates/paimon/src/table/vector_search_builder.rs:
##########
@@ -196,6 +197,33 @@ pub struct BatchVectorSearchBuilder<'a> {
options: HashMap<String, String>,
projection: Option<Vec<String>>,
filter: Option<Predicate>,
+ include_row_ids: Option<Arc<RoaringTreemap>>,
+ prepared_filter: Option<PreparedVectorSearchFilter>,
+}
+
+/// A scalar vector pre-filter resolved once against one pinned snapshot.
+///
+/// Reusing this value avoids repeating the same scalar-index/table read for
+/// every input batch of a lateral vector query.
+#[derive(Debug, Clone)]
+pub struct PreparedVectorSearchFilter {
+ table: Table,
+ include_row_ids: Arc<RoaringTreemap>,
+}
+
+impl PreparedVectorSearchFilter {
+ pub fn table(&self) -> &Table {
+ &self.table
+ }
+
+ pub fn include_row_ids(&self) -> &Arc<RoaringTreemap> {
+ &self.include_row_ids
+ }
+}
+
+fn same_vector_search_table(left: &Table, right: &Table) -> bool {
Review Comment:
[P2] Bind prepared filters to storage identity, not only URI text
Two supported public `Table::new` values can have the same normalized
location and branch while their `FileIO`s resolve that path to different
storage. For example, separate memory `FileIO` instances are intentionally
isolated (see `test_memory_operator_not_shared_between_file_io_instances`), and
`with_fs_operator` lets the operator root determine the actual filesystem.
Preparing on table A and attaching to table B therefore passes this check;
`execute` then switches entirely to the prepared table A and silently returns A
row IDs/scores. Please give a Table/FileIO clone family a stable private
lineage token, preserve it across table copies, and store/validate it in the
prepared filter (with a regression using two isolated FileIOs at the same
location).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
