Re: Disconnectedness in HNSW graphs in Lucene

2023-08-23 Thread Benjamin Trent
Nitiraj, Good experimentation! Connectedness within layers is indeed important. The algorithm itself should ensure connectedness of disjoint NSWs as it mutually connects nodes (selected over diversity). However, if the data is extremely clustered, this can cause connectedness to drop (few

Re: 8.11.3 release

2023-08-23 Thread Ishan Chattopadhyaya
Hi Jan, Yes, still targeting September. But I will slip on my initial plan of doing it by first week of September. I'm foreseeing mid September timeframe. Thanks for checking in. Regards, Ishan On Wed, 23 Aug, 2023, 5:05 pm Jan Høydahl, wrote: > Hi, > > Following up on Ishan's proposed 8.11.3

Re: 8.11.3 release

2023-08-23 Thread Jan Høydahl
Hi, Following up on Ishan's proposed 8.11.3 release (https://lists.apache.org/thread/3xjtv1sxqx8f9nvhkc0cb90b2p76nfx2) Does the Lucene project have any bugfix candidates for backporting? Ishan, are you still targeting September? Jan > 1. aug. 2023 kl. 14:57 skrev Ishan Chattopadhyaya : >

Disconnectedness in HNSW graphs in Lucene

2023-08-23 Thread Nitiraj Singh Rathore
Hi Lucene developers, I work for Amazon Retail Product search and we are using Lucene KNN for semantic search of products. We index product embeddings (vectors) into lucene (hnsw graph) and search them by generating query embedding at runtime. The product embeddings also receive regular updates