airborne12 commented on code in PR #68661:
URL: https://github.com/apache/doris/pull/68661#discussion_r4197842478
##########
be/src/exprs/function/match.cpp:
##########
@@ -382,42 +503,15 @@ Status
FunctionMatchPhrase::execute_match(FunctionContext* context, const std::s
auto current_src_array_offset = 0;
for (int i = 0; i < input_rows_count; i++) {
- auto data_tokens = analyse_data_token(column_name, analyzer_ctx,
string_col, i,
- array_offsets,
current_src_array_offset);
-
- // TODO: more efficient impl
- bool matched = false;
- auto data_it = data_tokens.begin();
- while (data_it != data_tokens.end()) {
- // find position of first token
- data_it =
- std::find_if(data_it, data_tokens.end(), [&](const
segment_v2::TermInfo& info) {
- return info.get_single_term() ==
query_tokens[0].get_single_term();
- });
- if (data_it != data_tokens.end()) {
- matched = true;
- auto data_it_next = ++data_it;
- auto query_it = query_tokens.begin() + 1;
- // compare query_tokens after the first to data_tokens one by
one
- while (query_it != query_tokens.end()) {
- if (data_it_next == data_tokens.end() ||
- data_it_next->get_single_term() !=
query_it->get_single_term()) {
- matched = false;
- break;
- }
- query_it++;
- data_it_next++;
- }
-
- if (matched) {
- break;
- }
+ const auto element_begin = array_offsets ? (*array_offsets)[i - 1] : i;
Review Comment:
Confirmed the cross-element mismatch.
`FunctionMatchTest.array_phrase_spans_non_null_elements` was RED for
MATCH_PHRASE, MATCH_PHRASE_PREFIX, and MATCH_PHRASE_EDGE on both `["hello",
"world"]` and `["hello", NULL, "world"]`. Commit `d9aeeb226c63` now streams
non-null elements through a query-length token window, so the fallback can
match adjacent terms across elements. The test is GREEN; `FunctionMatchTest.*`
passes 33/33, and the BE build passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
