github-actions[bot] commented on code in PR #69790:
URL: https://github.com/apache/doris/pull/69790#discussion_r4439039029
##########
be/src/storage/index/inverted/reader/collection_statistics.cpp:
##########
@@ -89,6 +98,23 @@ Result<std::shared_ptr<lucene::search::IndexSearcher>>
open_optional_clucene_sta
return std::make_shared<lucene::search::IndexSearcher>(reader, true);
}
+// Whether a SNII segment has no terms for this field: it indexed the field
with BKD, see
+// open_optional_clucene_stats_searcher, or the field is a VARIANT path this
segment does not hold,
+// as segments of {"a": 1} hold no path b. Any other missing index was not
built, and the caller
+// refuses to score the collection without it.
+Result<bool> snii_field_has_no_terms(const segment_v2::IndexFileReader&
file_reader,
+ const TabletIndex& index) {
+ const auto entries = DORIS_TRY(file_reader.snii_logical_indexes());
Review Comment:
[P2] Avoid copying the full SNII directory per scored field
`snii_logical_indexes()` returns an owning vector, while
`SniiSegmentReader::logical_indexes()` returns a const reference, so this call
deep-copies every directory entry, including suffix strings and BKD/ANN file
tables. `process_segment()` invokes this helper for each collected field in
every segment, including existing analyzed MATCH and SEARCH queries. On a SNII
table with a scoring `body` field and a VARIANT index containing many paths,
scoring `body` now copies all unrelated path metadata; scoring F fields repeats
that allocation/copying F times per segment. Use a presence/kind lookup under
the reader's existing lock, or obtain one directory snapshot per segment and
reuse it across fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
