github-actions[bot] commented on code in PR #68661:
URL: https://github.com/apache/doris/pull/68661#discussion_r4140589006
##########
be/src/exprs/function/match.cpp:
##########
@@ -241,9 +351,12 @@ inline std::vector<segment_v2::TermInfo>
FunctionMatchBase::analyse_data_token(
auto reader =
doris::segment_v2::inverted_index::InvertedIndexAnalyzer::create_reader(
analyzer_ctx->char_filter_map);
reader->init(str_ref.data, (int)str_ref.size, true);
- data_tokens =
+ auto element_tokens =
doris::segment_v2::inverted_index::InvertedIndexAnalyzer::get_analyse_result(
reader, analyzer_ctx->analyzer.get());
+ for (auto& token : element_tokens) {
Review Comment:
[P1] Skip NULL array elements when matching analyzed tokens. `IF(flag, NULL,
s)` can leave `s='alpha'` in the nested string while marking it null;
`array(...)` preserves that payload. With `flag=true`, `array(if(flag, NULL,
s), 'tail') MATCH_ANY 'alpha'` now returns true because this loop appends the
hidden `alpha`, although the indexed writer skips the null element and the base
fallback only saw `tail`. The new phrase/prefix/edge loops also visit that null
element. Pass the nested null map into both paths and exclude null elements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
