Hi devs, I'd like to initiate a discussion on PIP-47: Centroid-Based Sharding Design for IVF_PQ [1].
Paimon already supports IVF_PQ for vector search, but query-time pruning for centroid-based layouts is still limited. As a result, queries may read shards that are not actually relevant. This proposal introduces a routing-based pruning mechanism to narrow the search to relevant shards and improve query efficiency. Looking forward to hearing from you, thanks [1] https://cwiki.apache.org/confluence/spaces/PAIMON/pages/444334660/PIP-47+Centroid-Based+Sharding+Design+for+IVF_PQ Best, zhoufa
