JingsongLi opened a new pull request, #1492:
URL: https://github.com/apache/paimon-rust/pull/1492

   ### Purpose
   
   Part of #1490. Align range-sharded Data Evolution embedding-list indexes and 
MaxSim queries with apache/paimon#11290.
   
   ### Brief change log
   
   - Build cosine indexes over `ARRAY<VECTOR<FLOAT, d>>` and 
`ARRAY<ARRAY<FLOAT>>` by writing every non-null embedding with its physical row 
label.
   - Retrieve distinct row candidates per query vector, then score their union 
against complete embedding lists from the same snapshot. MaxSim sums the best 
cosine similarity for each query vector. Single vector queries and independent 
batches retain their existing result shapes.
   - Support the same scoring for raw fallback and scheduled search workers. 
Use double-precision accumulation on original coordinates for exact row scores.
   
   ### Tests
   
   The core MultiVector tests passed for both column types. They cover repeated 
row labels, null and empty lists, fixed dimensions, scheduled candidate 
refinement, extreme query scales, and raw fallback for a newly appended range.
   
   ### API and Format
   
   Adds `with_multi_vector` to the Rust and Python search builders and accepts 
a JSON matrix in the DataFusion `vector_search` table function. The index uses 
ordinary range manifests and repeated native row labels.
   
   ### Documentation
   
   Adds a query example, supported column types, and candidate refinement 
defaults. Draft for review of the distinct-row candidate retrieval policy.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to