Hi Alessandro, Thank you for the feedback. Kindly see my comments below,
*Ale*: https://www.elastic.co/blog/accelerating-vector-search-simd-instructions, I suggest to experiment with simD vector improvements (unless you are already doing it) * We will try this soon. * *Ale*: What about the machine memory? Following is the system specification: Linux ( CPU:64, RAM:488 GB, OS:Ubuntu 20.04.6 ) *Ale*: you can fine-tune the hyper-parameter to compromise a bit on recall in favour of performance (hnswBeamWidth, hnswMaxConnections) I am trying this as a first step. But I am sure it will impact recall. Regards, Iram Tariq | Software Architect NorthBay Direct: +1 (902) 329-7329 iram.ta...@northbaysolutions.net www.northbaysolutions.com On Thu, Mar 28, 2024 at 5:42 AM Alessandro Benedetti <a.benede...@sease.io> wrote: > That's interesting. > I think it's vital to get back some performance tests from the community. > Since my contribution to support Vector-search in Apache Solr was merged, > we got little or null feedback to understand its performance, in real-world > use cases. > Blogs, open benchmarks or even just this sort of mail message are welcome. > Let me reply in line: > -------------------------- > *Alessandro Benedetti* > Director @ Sease Ltd. > *Apache Lucene/Solr Committer* > *Apache Solr PMC Member* > > e-mail: a.benede...@sease.io > > > *Sease* - Information Retrieval Applied > Consulting | Training | Open Source > > Website: Sease.io <http://sease.io/> > LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter > <https://twitter.com/seaseltd> | Youtube > <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github > <https://github.com/seaseltd> > > > On Wed, 27 Mar 2024 at 21:06, Kent Fitch <kent.fi...@gmail.com> wrote: > > > Hi Iram, > > > > Is the machine doing lots of IO? If the hnsw graphs are not entirely in > > memory, performance will be poor. What JVM? You may get some benefit from > > simd support in java 21. Can you use the latest quantisation changes in > > Lucene to reduce memory footprint of the hnsw graphs? That's a large > topk, > > but I guess you need it? > > > > Best regards > > > > Kent Fitch > > > > On Thu, 28 Mar 2024, 5:12 am Iram Tariq, > > <iram.ta...@northbaysolutions.net.invalid> wrote: > > > > > Hi All, > > > > > > I am using Dense vectors in SOLR and facing slowness in it. Each search > > is > > > taking 10-25 seconds. I want to reduce the time to 5 seconds (or less > > > ideally). > > > > > > Following configurations are being used. > > > > > > > > > 1. *SOLR Version:* 9.3.0 > > > 2. *Lucene Version:* 9.7.0 > > > *Ale*: > https://www.elastic.co/blog/accelerating-vector-search-simd-instructions, > I > suggest to experiment with simD vector improvements (unless you are > already doing it) > > > > 3. *Vector Dimensions*: 384 > > > 4. *Total Shards:* 5 > > > 5. *Number of Vectors (Per shard*): 43209158 > > > 6. *JVM for each Instance:* 35GB > > > *Ale*: What about the machine memory? > > > > 7. *TopK: *1000 (Getting 1000 from each shard) > > > 8. *Rows: *1000 > > > 9. *Vector Field Schema: *<fieldType name="knn_vector_384" > > > class="solr.DenseVectorField" hnswMaxConnections="20" > > > knnAlgorithm="hnsw" > > > vectorDimension="384" similarityFunction="cosine" > hnswBeamWidth="40"/> > > > *Ale*: you can fine-tune the hyper-parameter to compromise a bit on recall > in favour of performance (hnswBeamWidth, hnswMaxConnections) > > > > 10. *Stored*: False > > > 11. *WebServer:* Apache Tomcat > > > 12. *System Specs*: Linux ( CPU:64, RAM:488 GB, OS:Ubuntu 20.04.6 ) > > > > > > Any sort of help/clue will be appreciated. > > > > > > > > > > > > Regards, > > > > > > > > > Iram Tariq | Software Architect > > > > > > NorthBay > > > > > > Direct: +1 (902) 329-7329 > > > > > > iram.ta...@northbaysolutions.net > > > > > > www.northbaysolutions.com > > > > > >
