JingsongLi opened a new pull request, #8302: URL: https://github.com/apache/paimon/pull/8302
## Summary Support vector search over unindexed raw rows when `global-index.search-mode` is configured as `full` or `detail`, while keeping the default `fast` mode index-only. This improves vector search freshness after new data is written before the vector global index is rebuilt. ## Changes - Add a `VectorGlobalIndexer` interface so vector index implementations can expose their metric for raw score computation. - Extend Java vector and batch vector reads to merge indexed results with raw rows outside indexed row-id ranges in `full` / `detail` modes. - Preserve Spark vector search behavior by using Spark execution for fast index-only mode and local raw fallback when raw rows need scanning. - Add Python support for `global-index.search-mode`, vector raw fallback, metric lookup, and raw row range pruning. - Add Java and Python tests covering default fast behavior, full-mode raw fallback, filtered raw fallback, and batch vector search. ## Testing - `mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -Dtest=VectorSearchBuilderTest test` - `mvn -pl paimon-common,paimon-core,paimon-lumina,paimon-vector,paimon-spark/paimon-spark-common -am -Pfast-build -DskipTests -DfailIfNoTests=false compile` - `mvn -pl paimon-common,paimon-core,paimon-lumina,paimon-vector,paimon-spark/paimon-spark-common -DskipTests spotless:check` - `python -m pytest paimon-python/pypaimon/tests/vector_search_filter_test.py -q` - `python -m pytest paimon-python/pypaimon/tests/global_index_update_action_test.py paimon-python/pypaimon/tests/table_scan_mode_test.py -q` ## Notes - `mvnflink` is not available in this local environment (`mvnflink not found`), so verification used Maven directly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
