rajat315315 opened a new issue, #16451: URL: https://github.com/apache/lucene/issues/16451
### Description ## Motivation Lucene heavily relies on DocValues for sorting, faceting, and range filtering. For byte-aligned packed formats (8-bit, 16-bit, and 32-bit), the bulk decoding process (`DocValuesBulkDecodeSupport`) decompresses compressed byte arrays back into native Java `long[]` value arrays. Currently, the default implementation (`DefaultDocValuesBulkDecodeSupport`) performs this decoding using scalar loops. While modern JVM HotSpot JIT (C2 compiler) tries to auto-vectorize loops, it frequently fails or generates suboptimal code for: 1. **Widening Type Casts**: Widening 8-bit bytes, 16-bit shorts, or 32-bit integers to 64-bit `long` lanes. 2. **Strided Memory Access**: Fetching compressed data using strided offset arithmetic. By leveraging the JDK Panama Vector API (`jdk.incubator.vector`), we can explicitly program vectorized instructions (using AVX2 or AVX-512 vector shapes depending on hardware support) to perform bulk decoding and widening operations in parallel vector lanes. --- ## Proposed Implementation Design The proposed implementation updates `PanamaDocValuesBulkDecodeSupport` to support hardware-accelerated vectorization for byte-aligned bit widths (8, 16, 32) using `LongVector` and statically imported zero-extending operators: * **8-Bit Packed Decoding (`decode8`)**: Loads a `ByteVector` and uses `.convertShape(ZERO_EXTEND_B2L, LongVector.SPECIES_PREFERRED, 0)` to widen byte lanes into long lanes in parallel. * **16-Bit Packed Decoding (`decode16`)**: Loads a `ByteVector`, reinterprets it as a `ShortVector` (`.reinterpretAsShorts()`), and widens it using `ZERO_EXTEND_S2L`. * **32-Bit Packed Decoding (`decode32`)**: Loads a `ByteVector`, reinterprets it as an `IntVector` (`.reinterpretAsInts()`), and widens it using `ZERO_EXTEND_I2L`. * **Tail Handling**: Safely bounds-checks the vector loop to process the maximum possible vector lanes without reading past array boundaries, falling back to the scalar decoder for the tail elements. * **Fallback**: Standard fallback to the scalar implementation in non-little-endian environments or when the platform's vector capabilities are insufficient (e.g., preferred species width is less than 32 bytes). --- ## Benchmark Results (Wikimedia 1M Dataset) We evaluated performance on a 1M document Wikimedia dataset using `luceneutil`, executing 5 JVM iterations with 20 warmups per query. The benchmark compared the clean standard `lucene_baseline` (`main` branch) against our candidate branch (`perf/simd-docvalues-decode`). Here are the averaged throughput (QPS) comparison results for the top-performing tasks: | Task / Query | Baseline QPS | Panama Candidate QPS | Throughput Change (Pct Diff) | Statistical Significance (p-value) | | :--- | :---: | :---: | :---: | :---: | | **Respell** (Fuzzy/Edit Distance) | 14.85 | 18.43 | **+24.1%** | 0.027 | | **LowPhrase** | 427.70 | 495.32 | **+15.8%** | 0.103 | | **Fuzzy1** | 34.47 | 39.51 | **+14.6%** | 0.150 | | **IntNRQ** (Numeric Range Query) | 208.01 | 232.59 | **+11.8%** | 0.186 | | **HighTermMonthSort** (Sorting) | 270.39 | 301.12 | **+11.4%** | 0.446 | | **MedIntervalsOrdered** | 37.82 | 40.30 | **+6.6%** | 0.473 | | **Prefix3** | 174.93 | 185.70 | **+6.2%** | 0.375 | | **HighPhrase** | 22.86 | 24.08 | **+5.3%** | 0.549 | | **LowIntervalsOrdered** | 62.04 | 64.47 | **+3.9%** | 0.612 | | **HighTermDayOfYearSort** (Sorting) | 100.37 | 104.13 | **+3.7%** | 0.712 | | **Fuzzy2** | 27.03 | 27.91 | **+3.2%** | 0.731 | ### Summary of Performance Impact * **Fuzzy Terms & Edit Distances**: Operations like `Respell` show a statistically significant **+24.1% QPS speedup** due to heavy DocValues decompression cycles. * **Sorting & Range Queries**: Sorting on compressed fields (`HighTermMonthSort` at **+11.4%**) and range filtering (`IntNRQ` at **+11.8%**) achieve substantial throughput improvements from parallel decompression. --- ## Current Status An active PR/branch `perf/simd-docvalues-decode` is currently working on this proposal. Correctness has been verified with a robust unit test suite (`TestDocValuesBulkDecodeSupport.java`) comparing default scalar outputs against Panama vector outputs across randomized bit widths, byte offsets, and alignment configurations. --- ## Code & Next Steps I have the draft code working in a local branch `perf/simd-docvalues-decode`. Before opening a pull request, I wanted to check if this approach looks reasonable to everyone. Specifically, I'd love feedback on: * Whether the vector widening casts (`ZERO_EXTEND_B2L`, etc.) are structured cleanly according to our current guidelines. * Any additional edge cases or platforms we should be testing. Uwe (@uschindler), Chris (@chegar999), Mike (@mikemccand), Adrien (@jpountz), Robert (@rmuir) — since you've worked on vector utility and decoding optimizations, I'd appreciate your thoughts on this! Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
