serhiy-bzhezytskyy opened a new pull request, #4648: URL: https://github.com/apache/solr/pull/4648
Follows up on the dev@ thread ["State of Lucene Index Sorting support in Solr"](https://lists.apache.org/thread/qlzqjw4mlf84n90mxbbml35x866gfk3w). **Purpose: this PR exists to make the *whole shape* of the index-sorting work reviewable in one place** — as I offered on the list ("open a PR with all changes to see the full picture"). It is **not** a request to merge all of this as one lump. I'm very happy to split it into per-ticket PRs, and to fold the config piece into Christine's existing work rather than land my own — see the note at the bottom. ## The picture (from David's "State of Lucene Index Sorting" report) Index sorting works today only through the indirect `SortingMergePolicyFactory` (a merge policy that does no merging, existing only to carry a `Sort` to `IndexWriterConfig.setIndexSort()`). This branch works through the open pieces end to end, in dependency order: | commit | ticket | what | |--------|--------|------| | SOLR-15390 | [15390](https://issues.apache.org/jira/browse/SOLR-15390) | use native `TopFieldCollector` early termination for `segmentTerminateEarly`; remove the deprecated `EarlyTerminatingSortingCollector` (Tomas said "feel free to take it") | | SOLR-13681 | [13681](https://issues.apache.org/jira/browse/SOLR-13681) | direct `<indexSort>` config in `<indexConfig>`, replacing the merge-policy indirection | | SOLR-12239 | [12239](https://issues.apache.org/jira/browse/SOLR-12239) | `RESORTINDEX` core-admin action to re-sort an existing index (also up standalone as #4644) | | SOLR-17310 | [17310](https://issues.apache.org/jira/browse/SOLR-17310) | `<segmentSort>` to configure Lucene's leaf sorter | | SOLR-12230 | [12230](https://issues.apache.org/jira/browse/SOLR-12230) | deprecate `SortingMergePolicy` now that `<indexSort>` replaces it | ## Credit (important — please read) - **SOLR-13681** reimplements the approach from **@cpoerschke's draft #313** on current `main` (the draft predates `MapWriter` and some config changes). Christine owns this lane; I'd rather help land #313 than land a competing change — this is here for the full picture, not to bypass her work. - **SOLR-17310** builds on **@weiwang19's #2477** (auto-closed as stale, not rejected). Credit for the leaf-sorter direction is his. (I stripped `Co-authored-by` trailers from the commits deliberately — I didn't want to attribute code to Christine or Wei publicly without asking them first. Crediting them here in prose instead; happy to add trailers back if they'd prefer.) ## Open design questions (for the list / reviewers) 1. **Config shape:** `<indexSort>` (within-segment) and `<segmentSort>` (between-segment) as *separate* `<indexConfig>` elements, vs. a unified `<indexSorters>` element. My reading of precedent leans separate (Lucene keeps `setIndexSort`/`setLeafSorter` distinct; ES/OpenSearch expose only the within-segment axis) — but no strong opinion. 2. **segmentTerminateEarly:** actually deprecate the param (15390's literal ask), or just modernize internals? This branch modernizes internals; the param-deprecation is left as a separate user-facing decision. ## Testing Unit + SolrCloud (~40 tests) pass locally, incl. `TestSegmentSorting` (`testSegmentTerminateEarly`, `testSegmentTerminateEarlyWithIndexSort`, `testAtomicUpdateOfSegmentSortField`) and `TestEarlyTerminatingQueries`. Compiles clean. ## What I'm asking Direction and collaboration — especially with @cpoerschke on 13681, since that's the keystone and her active work. If the shape looks right, I'll break this into reviewable per-ticket PRs (15390 first — it's self-contained). No rush. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
