JingsongLi opened a new pull request, #8459: URL: https://github.com/apache/paimon/pull/8459
## Summary Filter deleted rows out of global-index based vector and full-text search results without rebuilding existing index files. The search path now derives the current snapshot's live row ids from data files plus deletion vectors and uses that bitmap as an include filter for indexed search splits. ## Changes - Add a shared live-row filter for global-index search readers that subtracts deletion-vector positions from current data-file row-id ranges. - Apply the live-row include mask to indexed vector search, batch vector search, and indexed full-text search. - Add `includeRowIds` support to `FullTextSearch` and offset it correctly for split-local index readers. - Make the test full-text reader and Tantivy full-text reader respect full-text include row ids. - Add deletion-vector tests covering vector, batch vector, and full-text topK behavior after deletes. ## Testing - [x] `mvn -pl paimon-core -am -Pfast-build -Dtest=FullTextSearchBuilderTest,VectorSearchBuilderTest -DfailIfNoTests=false test` - [x] `mvn -pl paimon-core,paimon-common,paimon-tantivy/paimon-tantivy-index -am spotless:check -DskipTests` - [x] `mvn -pl paimon-core,paimon-common,paimon-tantivy/paimon-tantivy-index -am -DskipTests compile` ## Notes The table-level full-text path uses split-size candidate limits before applying include row ids, preserving topK correctness. Tantivy currently filters include row ids on the Java reader result because the JNI search API does not yet expose row-id filter pushdown. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
