airborne12 opened a new pull request, #68661: URL: https://github.com/apache/doris/pull/68661
### What problem does this PR solve? Without an inverted index, MATCH on an array of strings can silently miss rows when the matching token appears before the final array element. For example, ['alpha token', 'tail only'] fails to match MATCH_ANY 'alpha'. The array branch of FunctionMatchBase::analyse_data_token overwrites the accumulated tokens with each element's analyzer output, retaining only the final element's tokens. This change accumulates tokens from all array elements for MATCH_ANY, MATCH_ALL, and MATCH_REGEXP. MATCH_PHRASE, MATCH_PHRASE_PREFIX, and MATCH_PHRASE_EDGE evaluate each array element separately so phrases cannot span element boundaries. Their existing token-matching algorithms are extracted into file-local helpers. The new unit test covers all six MATCH variants, first and final elements, an empty final element, empty arrays, custom analyzers, keyword parsing, and phrase boundaries. ### Release note Fix missing matches in array MATCH queries evaluated without an inverted index, while preserving array element boundaries for phrase matching. ### Check List (For Author) - Test: Observed RED before the fix and GREEN afterward. Final ASAN FunctionMatchTest: 31/31 passed using `GLIBC_COMPATIBILITY=OFF bash run-be-ut.sh --run --filter='FunctionMatchTest.*' -j 32`. - Validation: clang-format 16, repository format checks, and `git diff --check` passed. - Validation limitations: The default ASAN UT configuration failed to link because Doris and the local LDB toolchain define duplicate glibc compatibility symbols. The compatibility override was used only for local UT execution. clang-tidy remains blocked by a pre-existing unmatched NOLINTEND in be/src/core/types.h:576; no remaining diagnostics were found on added or modified lines. No end-to-end regression cluster test was run. - Behavior changed: Yes. Array MATCH fallback retains tokens from every element, and phrase matching is confined to individual elements. - Does this need documentation: No; this restores expected query behavior. ### Scope Only be/src/exprs/function/match.cpp and be/test/exprs/function/function_match_test.cpp belong to this change (194 insertions, 110 deletions, including relocation of existing phrase-matching code). Exclude contrib/datasketches-cpp submodule changes and all local environment artifacts. Target repository: apache/doris. Target branch: master. The platform will commit, push to the fork, and create the PR after approval. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
