[
https://issues.apache.org/jira/browse/SOLR-18608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18133876#comment-18133876
]
Nick Shanin commented on SOLR-18608:
------------------------------------
Prepared fix branch:
[pr-5188-glob-field-loading-followup|https://github.com/nick-boss-tech/solr/tree/pr-5188-glob-field-loading-followup]
at {{555e1bf2398c08b1018ed44c056274c727bd6454}}, gated at that head (receipts
and run records are on file). The branch is based on PR 5188's head from before
it merged, so the incremental diff of the follow-up itself is five files: [PR
5188 production base to
branch|https://github.com/nick-boss-tech/solr/compare/db867d80365510790bb6a04ce33105cd196cf34c...555e1bf2398c08b1018ed44c056274c727bd6454].
Before a PR opens, the branch needs a rebase onto current main and a fresh
gate at the new head.
🤖 _AI text above_ 🤖 _(reviewed by Nick Shanin)_
> Honor glob field selection in real-time get and lazy document prefetch
> ----------------------------------------------------------------------
>
> Key: SOLR-18608
> URL: https://issues.apache.org/jira/browse/SOLR-18608
> Project: Solr
> Issue Type: Bug
> Reporter: Nick Shanin
> Priority: Major
>
> Partial-glob {{fl}} requests can load unrelated stored values in index-backed
> real-time get, and skip the lazy prefetch used by normal queries. The
> returned field list can still be correct; the gap is in stored-field loading.
> This follows [David Smiley's PR
> 5188|https://github.com/apache/solr/pull/5188], which has since merged, so
> this behavior is on main. The behavior was reproduced on its
> {{db867d80365510790bb6a04ce33105cd196cf34c}} production code using the same
> regression tests as the fix.
> h3. Reproduction
> Enable lazy field loading and the document cache. Index a document with
> stored {{id=7781}}, {{title=keyword}}, {{source_s=source}}, and
> {{unrelated_s=unrelated}}, then commit.
> * Request {{/get?id=7781&fl=tit*}}. The partial glob has no finite
> field-name set, so RTG passes {{null}} to the stored-field fetcher. Its
> document conversion also materializes fields already present in a cached
> Lucene document. Unrelated values are loaded even though they are excluded
> from the response. An unmatched glob such as {{fl=missing_*}} also returns an
> empty document while loading unrelated values.
> * Query {{/select?q=title:keyword&fl=source_*&hl=true&hl.fl=title}} with at
> most 50 results on a non-shard request. Lazy prefetch skips the glob request
> because {{getLuceneFieldNames()}} returns {{null}}. The equivalent explicit
> {{fl=source_s}} request prefetches response fields, highlight fields and the
> unique key.
> These loading states require inspection of the cached document before
> response serialization; response fields and snippets alone do not expose the
> problem.
> h3. Proposed change
> Share the existing stored-field selector with RTG and prefetch. Resolve glob
> matches and required transformer inputs before fetching. Filter RTG's cached
> fields before reading their values. When highlighting is enabled, add
> highlight fields and the unique key to the resolved prefetch selection.
> Preserve the existing non-shard and 50-result limits.
> h3. Verification
> On Linux with seed {{5188C0DE}}, four loading regression methods fail on the
> pre-fix production code at the expected assertions. All six focused methods
> pass at {{555e1bf2398c08b1018ed44c056274c727bd6454}}. The 24-row positive
> snippet matrix passes before and after the fix, including original and
> unified highlighting.
> The six full classes have 114 tests, zero failures, errors or skips:
> {{BasicFunctionalityTest}} 28, {{TestPseudoReturnFields}} 35,
> {{TestRetrieveFieldsOptimizer}} 1, {{ReturnFieldsTest}} 16,
> {{TestUnifiedSolrHighlighter}} 33 and {{LargeFieldTest}} 1. Tidy, Error Prone
> compilation and {{:solr:core:check -x test}} pass.
> Coverage includes unmatched globs, transformer inputs, alias collisions,
> reuse of cached documents, uncommitted RTG output, {{fl=*}}, pseudo-fields
> and 50/51-result and shard boundaries. No full core test run,
> whole-repository validation, disk-I/O measurement or latency benchmark was
> performed. Partial-glob term-vector behavior and docValues decoration are
> outside this change.
> 🤖 _AI text above_ 🤖 _(reviewed by Nick Shanin)_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
