JingsongLi opened a new pull request, #8203: URL: https://github.com/apache/paimon/pull/8203
## Summary Expose query-time vector search options through the Java and Python vector search APIs, and thread them into Flink, Spark, and Lumina search execution. This lets callers configure index-specific query parameters such as `ivf.nprobe` and `hnsw.ef_search` at search time. ## Changes - Add `options` storage to `VectorSearch` and builder APIs in Java and Python. - Thread options through core vector reads, Spark distributed vector reads, and Flink `vector_search` procedure execution. - Extend Spark `vector_search` TVF to accept optional map or semicolon-separated string options. - Merge Lumina base, index, and query options with query options taking precedence. - Add Java, Scala, and Python coverage for option propagation and merge precedence. ## Testing - `mvn -pl paimon-common -am -Pfast-build -DfailIfNoTests=false -Dtest=VectorSearchTest test` - `mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -Dtest=VectorSearchBuilderTest#testVectorSearchThreadsOptions test` - `mvn -pl paimon-lumina -am -Pfast-build -DfailIfNoTests=false -Dtest=LuminaVectorOptionsTest test` - `python -m pytest -q paimon-python/pypaimon/tests/vector_search_filter_test.py::VectorOptionsTest paimon-python/pypaimon/tests/vector_search_filter_test.py::LuminaOptionsTest paimon-python/pypaimon/tests/vector_search_filter_test.py::VectorSearchFilterTest::test_read_threads_options_to_vector_search` - `mvn -pl paimon-flink/paimon-flink-common -am -Pfast-build -DfailIfNoTests=false -Dtest=VectorSearchProcedureITCase#testVectorSearchWithOptions test` - `mvn -pl paimon-spark/paimon-spark-common -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=org.apache.paimon.spark.catalyst.plans.logical.VectorSearchQueryTest -Dtest=none test` - `mvn -pl paimon-spark/paimon-spark-3.2 -am -Pfast-build,spark3 -DfailIfNoTests=false -DwildcardSuites=org.apache.paimon.spark.PaimonScanBuilderTest -Dtest=none test` - `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
