[
https://issues.apache.org/jira/browse/SOLR-17892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ishan Chattopadhyaya updated SOLR-17892:
----------------------------------------
Description:
This issue proposes adding *cuVS-Lucene* as a pluggable codec in Solr to enable
GPU-accelerated vector indexing.
*Background*
* [cuVS-Lucene|https://github.com/rapidsai/cuvs-lucene] is a new NVIDIA
project that integrates GPU acceleration into Lucene for vector search.
* It supports building HNSW graphs on GPUs via the state-of-the-art *CAGRA*
algorithm.
* The first official release of cuVS-Lucene is planned for {*}early October{*}.
* At present, artifacts are not yet published on Maven Central. For early
adoption, development, and testing, SearchScale is publishing temporary
artifacts to its Maven repository. Once released, official artifacts will be
available on Maven Central, and the PR will be updated to use the released
artifact accordingly.
*Usage in Solr*
This change introduces the ability to configure Solr to use the
{{*Lucene101AcceleratedHNSWVectorsFormat*}} provided by the cuVS-Lucene. Users
can opt to use the GPU-accelerated indexing by selecting the {{CuVSCodec}}
codec, while retaining compatibility with existing CPU-based codecs.
Documentation will include steps on how to enable and configure this format
within Solr. Prerequisites includes a
*Testing Strategy*
* NVIDIA is dedicating GPU resources for testing cuVS-Lucene directly.
* In Solr’s test framework:
** Tests for cuVS-Lucene will be skipped on non-GPU machines.
** On GPU-enabled machines, the tests will run fully, validating integration.
* We are also exploring the possibility of contributing GPU resources to
Apache Solr’s CI infrastructure for continuous GPU test coverage (to be
discussed further with the community).
*Benchmarks*
We will publish benchmark results shortly to demonstrate the performance
improvements of GPU-accelerated HNSW graph construction compared to CPU-only
implementations.
*Motivation for Solr 10*
Including cuVS-Lucene support in *Solr 10* would highlight Solr’s adoption of
first-class GPU acceleration, delivering significant performance improvements
for vector search and positioning Solr as a leader in large-scale AI/ML search
workloads.
was:
This issue proposes adding *cuVS-Lucene* as a pluggable codec in Solr to enable
GPU-accelerated vector indexing.
*Background*
* [cuVS-Lucene|https://github.com/rapidsai/cuvs-lucene] is a new NVIDIA
project that integrates GPU acceleration into Lucene for vector search.
* It supports building HNSW graphs on GPUs via the state-of-the-art *CAGRA*
algorithm.
* The first official release of cuVS-Lucene is planned for {*}early October{*}.
* At present, artifacts are not yet published on Maven Central. For early
adoption, development, and testing, SearchScale is publishing temporary
artifacts to its Maven repository. Once released, official artifacts will be
available on Maven Central, and the PR will be updated to use the released
artifact accordingly.
*Usage in Solr*
This change introduces the ability to configure Solr to use the
{{*Lucene101AcceleratedHNSWVectorsFormat*}} provided by the cuVS-Lucene. Users
can opt to use the GPU-accelerated indexing by selecting the {{CuVSCodec}}
codec, while retaining compatibility with existing CPU-based codecs.
Documentation will include steps on how to enable and configure this format
within Solr.
*Testing Strategy*
* NVIDIA is dedicating GPU resources for testing cuVS-Lucene directly.
* In Solr’s test framework:
** Tests for cuVS-Lucene will be skipped on non-GPU machines.
** On GPU-enabled machines, the tests will run fully, validating integration.
* We are also exploring the possibility of contributing GPU resources to
Apache Solr’s CI infrastructure for continuous GPU test coverage (to be
discussed further with the community).
*Benchmarks*
We will publish benchmark results shortly to demonstrate the performance
improvements of GPU-accelerated HNSW graph construction compared to CPU-only
implementations.
*Motivation for Solr 10*
Including cuVS-Lucene support in *Solr 10* would highlight Solr’s adoption of
first-class GPU acceleration, delivering significant performance improvements
for vector search and positioning Solr as a leader in large-scale AI/ML search
workloads.
> Add support for cuVS-Lucene as a pluggable codec in Solr
> --------------------------------------------------------
>
> Key: SOLR-17892
> URL: https://issues.apache.org/jira/browse/SOLR-17892
> Project: Solr
> Issue Type: New Feature
> Components: vector-search
> Reporter: Vivek Narang
> Priority: Major
> Labels: pull-request-available
> Fix For: main (10.0)
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This issue proposes adding *cuVS-Lucene* as a pluggable codec in Solr to
> enable GPU-accelerated vector indexing.
> *Background*
> * [cuVS-Lucene|https://github.com/rapidsai/cuvs-lucene] is a new NVIDIA
> project that integrates GPU acceleration into Lucene for vector search.
> * It supports building HNSW graphs on GPUs via the state-of-the-art *CAGRA*
> algorithm.
> * The first official release of cuVS-Lucene is planned for {*}early
> October{*}.
> * At present, artifacts are not yet published on Maven Central. For early
> adoption, development, and testing, SearchScale is publishing temporary
> artifacts to its Maven repository. Once released, official artifacts will be
> available on Maven Central, and the PR will be updated to use the released
> artifact accordingly.
>
> *Usage in Solr*
> This change introduces the ability to configure Solr to use the
> {{*Lucene101AcceleratedHNSWVectorsFormat*}} provided by the cuVS-Lucene.
> Users can opt to use the GPU-accelerated indexing by selecting the
> {{CuVSCodec}} codec, while retaining compatibility with existing CPU-based
> codecs. Documentation will include steps on how to enable and configure this
> format within Solr. Prerequisites includes a
>
> *Testing Strategy*
> * NVIDIA is dedicating GPU resources for testing cuVS-Lucene directly.
> * In Solr’s test framework:
> ** Tests for cuVS-Lucene will be skipped on non-GPU machines.
> ** On GPU-enabled machines, the tests will run fully, validating integration.
> * We are also exploring the possibility of contributing GPU resources to
> Apache Solr’s CI infrastructure for continuous GPU test coverage (to be
> discussed further with the community).
>
> *Benchmarks*
> We will publish benchmark results shortly to demonstrate the performance
> improvements of GPU-accelerated HNSW graph construction compared to CPU-only
> implementations.
> *Motivation for Solr 10*
> Including cuVS-Lucene support in *Solr 10* would highlight Solr’s adoption of
> first-class GPU acceleration, delivering significant performance improvements
> for vector search and positioning Solr as a leader in large-scale AI/ML
> search workloads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
