airborne12 commented on code in PR #65549:
URL: https://github.com/apache/doris/pull/65549#discussion_r3619024834
##########
be/src/exprs/function/function_search.cpp:
##########
@@ -412,42 +432,30 @@ Status
FunctionSearch::evaluate_inverted_index_with_search_param(
}
std::shared_ptr<roaring::Roaring> roaring =
std::make_shared<roaring::Roaring>();
+ std::shared_ptr<roaring::Roaring> null_bitmap = nullptr;
{
int64_t exec_dummy = 0;
+ const bool is_top_k = enable_scoring && !is_asc && top_k > 0;
SCOPED_RAW_TIMER(stats ?
&stats->inverted_index_searcher_search_exec_timer : &exec_dummy);
- if (enable_scoring && !is_asc && top_k > 0) {
+ if (is_top_k) {
bool use_wand = index_query_context->runtime_state != nullptr &&
index_query_context->runtime_state->query_options()
.enable_inverted_index_wand_query;
+ null_bitmap = extract_query_null_bitmap(weight, exec_ctx,
root_binding_key);
Review Comment:
[blocking] This reads the NULL bitmap from a fresh scoring scorer before it
has traversed all documents. For `OP_NOT`, `OperatorBooleanWeight<SumCombiner>`
builds an `AndNotScorer`, whose `_null_bitmap` is populated incrementally by
`_advance_to()`; `has_null_bitmap()` / `get_null_bitmap()` do not force
completion.
With multiple sub-readers, consider local doc 0 = TRUE/non-NULL and local
doc 1 = NULL. The constructor stops at doc 0, so this extraction misses doc 1.
Because this PR also removes `null_resolver` from segment contexts, the segment
scorer can then emit doc 1 as TRUE; both `excluded_docs` and the final
`mask_out_null()` use the same incomplete bitmap. A NULL/UNKNOWN row can
therefore be returned and occupy a top-k slot.
I compiled a minimal probe against this PR HEAD and observed:
```text
initial_doc=0 null_before=0
after_advance_doc=2 null_after=1
```
Please materialize NULL semantics through a complete non-scoring
three-valued evaluation, or provide a segment-local resolver with the global
bitmap sliced/shifted by `seg_base`. Please also add a multi-sub-reader
pure-NOT test with a non-leading NULL for WAND on/off and the scoring non-top-k
path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
