CH-Abhinav opened a new pull request, #16723:
URL: https://github.com/apache/lucene/pull/16723
### Description
Addresses item 1 in #15287: allow Faiss codec to use SIMD instructions at
runtime.
Previously, `FaissLibrary.java` and `FaissNativeWrapper.java` hardcoded
`System.loadLibrary("faiss_c")`. In Conda/PyTorch, `libfaiss_c.so` dynamically
links against the generic, non-vectorized `libfaiss.so` instead of SIMD
alternatives even if they exist.
### Changes
1. **CPU Feature Visibility in `Constants.java`**:
- Made `HAS_AVX2` and `HAS_SVE` public.
- Added `HAS_AVX512` based on HotSpot's `UseAVX >= 3`.
2. **Dynamic Fallback in `FaissNativeWrapper.java`**:
- Allows explicit override via system property
`-Dlucene.faiss.libname=<name>`.
- On `amd64` / `x86_64`: attempts loading `faiss_c_avx512` $\rightarrow$
`faiss_c_avx2` $\rightarrow$ generic `faiss_c`.
- On `aarch64`: attempts loading `faiss_c_sve` $\rightarrow$ generic
`faiss_c`.
- Records the successfully loaded library name in
`FaissNativeWrapper.LOADED_LIBRARY`.
3. **CI Testing (`run-special-checks-sandbox.yml`)**:
- Added a step to create `libfaiss_c_avx2.so` using `patchelf
--replace-needed libfaiss.so libfaiss_avx2.so` in the CI conda environment.
- Ensures the sandbox CI test suite actually validates and exercises the
AVX2 build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
