lxy-9602 commented on PR #11249:
URL: https://github.com/apache/paimon/pull/11249#issuecomment-7074545677

   Hi, @JingsongLi Thank you for this contribution! Centroid-based sharding is 
a valuable improvement for reducing query fan-out across index shards, 
especially under high QPS.
   
   I wonder would it make sense to make the implementation more 
backend-independent? For example, we could have a shared centroid build and 
query layer responsible for training the router, assigning vectors to 
centroids, and selecting shards at query time. Each shard’s writer and reader 
could then be created through a backend-specific factory, supporting Native 
IVF/DiskANN as well as Lumina.
   
   This would let other vector index backends reuse the routing layer while 
retaining their own indexing and search implementations. What do you think 
about this separation?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to