nick-boss-tech opened a new pull request, #5149: URL: https://github.com/apache/solr/pull/5149
🤖 *AI text below* 🤖 *(posted on behalf of Nick Shanin)* https://issues.apache.org/jira/browse/SOLR-9595 ## What happens today **Each call to terms() or getLiveDocs() on a SlowCompositeReaderWrapper builds a new merged view.** On the merge base, [cabedd1d9680](https://github.com/nick-boss-tech/solr/blob/cabedd1d968059215188f4e7563fb303241899ed/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L154-L158), `terms(field)` calls `MultiTerms.getTerms` on every call. [getLiveDocs()](https://github.com/nick-boss-tech/solr/blob/cabedd1d968059215188f4e7563fb303241899ed/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L352-L356) calls `MultiBits.getLiveDocs` on every call, with a "TODO cache?" note. On the base code, both return a new object each time, so the merge work repeats. ## What this change does **Terms are cached per field. The live-docs view is built once, when the wrapper is created.** - Merged terms are kept in a map keyed by field name ([L86](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L86), [terms(), L161-L172](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L161-L172)). A field with no terms is not stored, so it is looked up again on the next call. - The live-docs view is built in the constructor ([L88](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L88), [L134](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L134)) and returned by [getLiveDocs(), L366-L370](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L366-L370). - A searcher makes one wrapper for its reader ([SolrIndexSearcher, L400](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L400)), so the cached views last as long as that searcher. ## Proof **On the base code, the new test fails because two calls return different objects. With the change, the class passes 3 of 3.** - [TestSlowCompositeReaderWrapper](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/test/org/apache/solr/index/TestSlowCompositeReaderWrapper.java#L135-L173) 3 of 3 at `bf99cd0e89c255889b3d801e2ec863813dad78a7`, verified 2026-10-10. The new test, `testTermsAndLiveDocsAreCached`, keeps two segments with live documents, so the wrapper has more than one piece to merge. - On the base code the class fails on the check that two calls return the same object, for terms or live docs. It does not fail at the setup check. ## Limits **Doc values are not cached, the live-docs view is built eagerly, and no timing is measured.** - The ticket asks to cache the Multi* objects in general, and names MultiDocValues. This change covers terms and live docs only. Numeric, binary, sorted numeric and norm doc values are still built on each call ([L177](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L177), [L183](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L183), [L189](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L189), [L339](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/solr/core/src/java/org/apache/solr/index/SlowCompositeReaderWrapper.java#L339)). A follow-up ticket and PR for those can be opened on request. - The live-docs view is built in the constructor, so every wrapper pays for it, even when no caller asks for live docs. A lazy build would avoid that cost. It is not done here. - No timing is measured. This change is about repeated object construction, not speed. - Only TestSlowCompositeReaderWrapper is run for this description. Changelog: [`changelog/unreleased/SOLR-9595-cache-slow-reader-terms.yml`](https://github.com/nick-boss-tech/solr/blob/bf99cd0e89c255889b3d801e2ec863813dad78a7/changelog/unreleased/SOLR-9595-cache-slow-reader-terms.yml) ### AI assistance AI agents assisted with research, implementation, review, and drafting. Nick Shanin directed the work and takes responsibility for this contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
