[
https://issues.apache.org/jira/browse/SOLR-17892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18029970#comment-18029970
]
Vivek Narang commented on SOLR-17892:
-------------------------------------
{quote}If/when (and it's only a matter of time) classes like
Lucene99FlatVectorsFormat , Lucene99HnswVectorsReader, and/or
Lucene99HnswVectorsWriter get replaced by newer/better versions, and move to
the backward_codec namespace in Lucene-10.Z, Solr will be unable to upgrade to
Lucene-10.Z unless/until cuvs-lucene comes out with a new "10.Z compatible"
version.
{quote}
I am trying an approach using the Java Reflection API to solve this and will
provide more details soon. Thanks!
> Add support for cuVS-Lucene as a pluggable codec in Solr
> --------------------------------------------------------
>
> Key: SOLR-17892
> URL: https://issues.apache.org/jira/browse/SOLR-17892
> Project: Solr
> Issue Type: New Feature
> Components: vector-search
> Reporter: Vivek Narang
> Assignee: Ishan Chattopadhyaya
> Priority: Major
> Labels: pull-request-available
> Fix For: main (10.0)
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> This issue proposes adding *cuVS-Lucene* as a pluggable codec in Solr to
> enable GPU-accelerated vector indexing.
> *Background*
> * [cuVS-Lucene|https://github.com/rapidsai/cuvs-lucene] is a new NVIDIA
> project that integrates GPU acceleration into Lucene for vector search.
> * It supports building HNSW graphs on GPUs via the state-of-the-art *CAGRA*
> algorithm.
> * The first official release of cuVS-Lucene is planned for {*}early
> October{*}.
> * At present, artifacts are not yet published on Maven Central. For early
> adoption, development, and testing, SearchScale is publishing temporary
> artifacts to its Maven repository. Once released, official artifacts will be
> available on Maven Central, and the PR will be updated to use the released
> artifact accordingly.
>
> *Usage in Solr*
> This change introduces the ability to configure Solr to use the
> {{*Lucene101AcceleratedHNSWVectorsFormat*}} provided by the cuVS-Lucene.
> Users can opt to use the GPU-accelerated indexing by selecting the
> {{CuVSCodec}} codec, while retaining compatibility with existing CPU-based
> codecs. Documentation will include steps on how to enable and configure this
> format within Solr. Prerequisites includes a
>
> *Testing Strategy*
> * NVIDIA is dedicating GPU resources for testing cuVS-Lucene directly.
> * In Solr’s test framework:
> ** Tests for cuVS-Lucene will be skipped on non-GPU machines.
> ** On GPU-enabled machines, the tests will run fully, validating integration.
> * We are also exploring the possibility of contributing GPU resources to
> Apache Solr’s CI infrastructure for continuous GPU test coverage (to be
> discussed further with the community).
>
> *Benchmarks*
> We will publish benchmark results shortly to demonstrate the performance
> improvements of GPU-accelerated HNSW graph construction compared to CPU-only
> implementations.
> *Motivation for Solr 10*
> Including cuVS-Lucene support in *Solr 10* would highlight Solr’s adoption of
> first-class GPU acceleration, delivering significant performance improvements
> for vector search and positioning Solr as a leader in large-scale AI/ML
> search workloads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
