Definitely cool Ishan & team! I think it's super clever to do this at a Lucene codec level. I'm surprised Luxir is able to deliver impressive gains in this way in spite of "only" being used as a codec. I also saw Yonik's Luxir presentation on Monday.
Do you plan on contributing the Solr patches by themselves, assuming they are each appropriate? I didn't look at them in detail. Your descriptions suggested utility without Luxir. On Fri, Oct 23, 2026 at 3:23 PM Ishan Chattopadhyaya < [email protected]> wrote: > Hi all, > > For years, Solr has been only as fast as Lucene allows. Every gain in > storage formats, query execution or faceting has had to come from upstream, > on Lucene's timeline and within the limits of the JVM. At SearchScale, we > wanted to see how far Solr could go without waiting on that, so we've > started exploring what it would take to swap Lucene's internals for > something faster: Luxir <https://github.com/luxir-search/luxir/>, a C++ > search engine, by Yonik Seeley! > > SoLux runs Luxir at the codec level, and the split works like this: > > - *Solr and Lucene still handle:* documents, analysis, segments, > commits, NRT and SolrCloud. > - *Luxir takes over:* storing the postings, doc values, points and > stored fields, and running queries, filters, facets and joins natively > in > one call per request, through the FFM API, with no JNI. > - *Packaging:* it ships as a Solr module (luxir-codec), enabled through > the configset. > > *Headline numbers.* We indexed 100M e-commerce events into 3 nodes × 3 > shards, then ran four query types from 16 concurrent clients, on a Ryzen 9 > 7950X and a Ryzen 9 5950X: > > SoLux vs stock Solr > Indexing 1.8–2.1× faster > Terms facet throughput 2.0–2.1× > Range facet throughput 2.7–2.8× > Join throughput 5.9–6.0× > All query tasks, total time 2.1× less > product_id terms facet + avg(price) 0.55–0.57×, slower: our next target > > The benchmark, full results, exact versions and step-by-step instructions > to reproduce it are in luxir.md. It also lists the changes behind these > numbers and what each one contributed: > https://github.com/SearchScale/solr-bench/blob/master/luxir.md > > *Changes to Luxir.* > > SoLux runs Luxir with four patches on top of upstream Luxir project, kept > in luxir-lucene. These are needed to improve on query and indexing > performance. > > - 0001 > < > https://github.com/SearchScale/luxir-lucene/blob/main/luxir-patches/0001-query-materialize-large-point-ranges-as-bitsets.patch > >: > build large point ranges as bitsets instead of sorting doc arrays. > - 0002 > < > https://github.com/SearchScale/luxir-lucene/blob/main/luxir-patches/0002-query-skip-term-windows-without-docs-when-counting.patch > >: > when counting, jump to the window holding a term's next doc instead of > stepping through every window. > - 0003 > < > https://github.com/SearchScale/luxir-lucene/blob/main/luxir-patches/0003-reader-seek-explicit-term-sets-block-by-block.patch > >: > look up explicit term sets block by block, decoding only the requested > terms' metadata. > - 0004 > < > https://github.com/SearchScale/luxir-lucene/blob/main/luxir-patches/0004-query-answer-sparse-range-verification-from-block-re.patch > >: > answer a range check behind a selective filter from each column block's > min > and max, reading values only in blocks that cross the range. Together > with > a faster range-facet bucket lookup, this took filtered range facets from > 14.2 to 5.3 ms per shard. > > *Caveats:* > > - This is experimental. Each configuration ran once. > - The Luxir nodes run with the security manager off, so the native > library can load. > - SoLux still sits on Lucene's segments and needs four small Solr core > changes. > > I'd love your feedback, and results from your own hardware. > > Thanks, > Ishan >
