[
https://issues.apache.org/jira/browse/PHOENIX-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18125781#comment-18125781
]
Andrew Kyle Purtell edited comment on PHOENIX-8009 at 10/10/26 8:26 AM:
------------------------------------------------------------------------
I built a HNSW Graph-Based Vector Index based on JVector + MOB Storage that was
successful, but was a GLOBAL index.
The design is mostly reusable as a LOCAL index , and as a LOCAL index that will
be a better design because the JVector segments have the same keyspace
partitioning and lifecycle as the data table regions. A refactor is underway.
Updated design doc and PR coming soon.
was (Author: apurtell):
I built a HNSW Graph-Based Vector Index based on JVector + MOB Storage that was
successful, but was a GLOBAL index.
The design is mostly reusable as a LOCAL index , and as a LOCAL index that will
be a better design because the JVector segments have the same keyspace
partitioning and lifecycle as the data table regions, so really should be
implemented in the local index style. A refactor of that is underway. Updated
design doc and PR coming soon.
> Vector Indexes Phase 2
> ----------------------
>
> Key: PHOENIX-8009
> URL: https://issues.apache.org/jira/browse/PHOENIX-8009
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: Andrew Kyle Purtell
> Assignee: Andrew Kyle Purtell
> Priority: Major
> Fix For: 5.4.0
>
>
> The IVF algorithm delivered in PHOENIX-7998 provides effective approximate
> nearest-neighbor search by partitioning high dimensional space into Voronoi
> cells, but its cluster-centric design imposes recall limitations under high
> dimensional skew and requires periodic centroid retraining as data
> distributions shift. Hierarchical Navigable Small World (HNSW) graph indexes
> can deliver higher recall at equivalent query latency and achieve two-fold to
> five-fold recall improvements over IVF at equivalent latency budgets by
> introducing a graph-based ANN algorithm alongside the existing IVF algorithm.
> This capability targets similarity search queries where high recall is
> paramount, update heavy workloads where IVF centroid drift degrades quality
> between rebuild cycles, and moderate scale vector collections where the
> memory footprint remains tractable.
> Design doc: https://gist.github.com/apurtell/e4668cddf949c43955d4a919489d4228
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
