That's interesting. I think it's vital to get back some performance tests from the community. Since my contribution to support Vector-search in Apache Solr was merged, we got little or null feedback to understand its performance, in real-world use cases. Blogs, open benchmarks or even just this sort of mail message are welcome. Let me reply in line: -------------------------- *Alessandro Benedetti* Director @ Sease Ltd. *Apache Lucene/Solr Committer* *Apache Solr PMC Member*
e-mail: a.benede...@sease.io *Sease* - Information Retrieval Applied Consulting | Training | Open Source Website: Sease.io <http://sease.io/> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter <https://twitter.com/seaseltd> | Youtube <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github <https://github.com/seaseltd> On Wed, 27 Mar 2024 at 21:06, Kent Fitch <kent.fi...@gmail.com> wrote: > Hi Iram, > > Is the machine doing lots of IO? If the hnsw graphs are not entirely in > memory, performance will be poor. What JVM? You may get some benefit from > simd support in java 21. Can you use the latest quantisation changes in > Lucene to reduce memory footprint of the hnsw graphs? That's a large topk, > but I guess you need it? > > Best regards > > Kent Fitch > > On Thu, 28 Mar 2024, 5:12 am Iram Tariq, > <iram.ta...@northbaysolutions.net.invalid> wrote: > > > Hi All, > > > > I am using Dense vectors in SOLR and facing slowness in it. Each search > is > > taking 10-25 seconds. I want to reduce the time to 5 seconds (or less > > ideally). > > > > Following configurations are being used. > > > > > > 1. *SOLR Version:* 9.3.0 > > 2. *Lucene Version:* 9.7.0 > *Ale*: https://www.elastic.co/blog/accelerating-vector-search-simd-instructions, I suggest to experiment with simD vector improvements (unless you are already doing it) > > 3. *Vector Dimensions*: 384 > > 4. *Total Shards:* 5 > > 5. *Number of Vectors (Per shard*): 43209158 > > 6. *JVM for each Instance:* 35GB > *Ale*: What about the machine memory? > > 7. *TopK: *1000 (Getting 1000 from each shard) > > 8. *Rows: *1000 > > 9. *Vector Field Schema: *<fieldType name="knn_vector_384" > > class="solr.DenseVectorField" hnswMaxConnections="20" > > knnAlgorithm="hnsw" > > vectorDimension="384" similarityFunction="cosine" hnswBeamWidth="40"/> > *Ale*: you can fine-tune the hyper-parameter to compromise a bit on recall in favour of performance (hnswBeamWidth, hnswMaxConnections) > > 10. *Stored*: False > > 11. *WebServer:* Apache Tomcat > > 12. *System Specs*: Linux ( CPU:64, RAM:488 GB, OS:Ubuntu 20.04.6 ) > > > > Any sort of help/clue will be appreciated. > > > > > > > > Regards, > > > > > > Iram Tariq | Software Architect > > > > NorthBay > > > > Direct: +1 (902) 329-7329 > > > > iram.ta...@northbaysolutions.net > > > > www.northbaysolutions.com > > >
