Nick Shanin created SOLR-18608:
----------------------------------
Summary: Honor glob field selection in real-time get and lazy
document prefetch
Key: SOLR-18608
URL: https://issues.apache.org/jira/browse/SOLR-18608
Project: Solr
Issue Type: Bug
Reporter: Nick Shanin
Partial-glob {{fl}} requests can load unrelated stored values in index-backed
real-time get, and skip the lazy prefetch used by normal queries. The returned
field list can still be correct; the gap is in stored-field loading.
This follows [David Smiley's PR 5188|https://github.com/apache/solr/pull/5188],
which has since merged, so this behavior is on main. The behavior was
reproduced on its {{db867d80365510790bb6a04ce33105cd196cf34c}} production code
using the same regression tests as the fix.
h3. Reproduction
Enable lazy field loading and the document cache. Index a document with stored
{{id=7781}}, {{title=keyword}}, {{source_s=source}}, and
{{unrelated_s=unrelated}}, then commit.
* Request {{/get?id=7781&fl=tit*}}. The partial glob has no finite field-name
set, so RTG passes {{null}} to the stored-field fetcher. Its document
conversion also materializes fields already present in a cached Lucene
document. Unrelated values are loaded even though they are excluded from the
response. An unmatched glob such as {{fl=missing_*}} also returns an empty
document while loading unrelated values.
* Query {{/select?q=title:keyword&fl=source_*&hl=true&hl.fl=title}} with at
most 50 results on a non-shard request. Lazy prefetch skips the glob request
because {{getLuceneFieldNames()}} returns {{null}}. The equivalent explicit
{{fl=source_s}} request prefetches response fields, highlight fields and the
unique key.
These loading states require inspection of the cached document before response
serialization; response fields and snippets alone do not expose the problem.
h3. Proposed change
Share the existing stored-field selector with RTG and prefetch. Resolve glob
matches and required transformer inputs before fetching. Filter RTG's cached
fields before reading their values. When highlighting is enabled, add highlight
fields and the unique key to the resolved prefetch selection. Preserve the
existing non-shard and 50-result limits.
h3. Verification
On Linux with seed {{5188C0DE}}, four loading regression methods fail on the
pre-fix production code at the expected assertions. All six focused methods
pass at {{555e1bf2398c08b1018ed44c056274c727bd6454}}. The 24-row positive
snippet matrix passes before and after the fix, including original and unified
highlighting.
The six full classes have 114 tests, zero failures, errors or skips:
{{BasicFunctionalityTest}} 28, {{TestPseudoReturnFields}} 35,
{{TestRetrieveFieldsOptimizer}} 1, {{ReturnFieldsTest}} 16,
{{TestUnifiedSolrHighlighter}} 33 and {{LargeFieldTest}} 1. Tidy, Error Prone
compilation and {{:solr:core:check -x test}} pass.
Coverage includes unmatched globs, transformer inputs, alias collisions, reuse
of cached documents, uncommitted RTG output, {{fl=*}}, pseudo-fields and
50/51-result and shard boundaries. No full core test run, whole-repository
validation, disk-I/O measurement or latency benchmark was performed.
Partial-glob term-vector behavior and docValues decoration are outside this
change.
🤖 _AI text above_ 🤖 _(reviewed by Nick Shanin)_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
