airborne12 opened a new pull request, #69206: URL: https://github.com/apache/doris/pull/69206
### What problem does this PR solve? Issue Number: None Related PR: #68758, #67180, #67538, #67918 Problem Summary: Backport #68758 to branch-4.2. MATCH and SEARCH now lower to a shared logical query engine for CLucene and SNII indexes. The backport preserves branch-4.2 APIs for scan-schema column binding, segment row counts, rowset segment traversal, and BKD type conversion. It passes the segment delete bitmap to the SEARCH Top-K collector so deleted rows do not consume the requested result count. Source squash commit: `14fde1c1dc096f09c17af019c5e0cff3175da6c9` (cherry-picked with `-x`). Branch-4.2 has no Kuromoji implementation, so the source-only Kuromoji code and metadata are omitted. The [complete hunk audit](https://github.com/airborne12/apache-doris/blob/pick-68758-branch-4.2/docs/backport-68758-hunk-audit.md) accounts for all 2,641 source hunks: | Disposition | Source hunks | | --- | ---: | | Ported with byte-identical file post-image | 1,528 | | Ported file deletion | 124 | | Ported hunk post-image in branch-different files | 886 | | Adapted to branch-4.2 APIs | 7 | | Not applicable, with target evidence | 96 | `git range-diff` against the source squash commit was reviewed. The material differences are the branch-4.2 scan-schema and type API adaptations, direct includes required after the source-tree move, preservation of the branch-only `RowIdStorageReader` test, and omission of Kuromoji and build-hygiene paths that do not exist on branch-4.2. The audit gives the disposition of every source hunk. The 1,036 non-Latin code literals in the PR diff are tokenizer, query, and Unicode test input or expected data. Each is bound to an exact finding ID in the shared pre-push English check. Comments, logs, errors, and diagnostics remain English. ### Release note MATCH and SEARCH use one query engine across V2 and SNII inverted indexes. SEARCH regexp, prefix, wildcard, phrase, scoring, and NULL behavior align across both formats as described in #68758. ### Check List (For Author) - Test - [x] Regression test: all five source suites passed on an isolated branch-4.2 FE/BE cluster - [x] Unit Test: 1,045 focused BE tests passed; all 461 enabled source-added tests executed - [x] Manual test: isolated FE/BE started, backend registered and alive, and `SELECT 1 AS ready` succeeded - Behavior changed: - [x] Yes. MATCH and SEARCH share query execution across V2 and SNII indexes. - Does this need documentation? - [x] No. ### Validation - `./build-support/check-format.sh`: PASS. - `./build.sh --be -j64` (ASAN): PASS. - `./build.sh --fe -j8` (JDK 17, Checkstyle enabled): PASS. - `./run-be-ut.sh --run --filter=... -j64` (ASAN): 1,045 tests passed in 74 suites; all 461 enabled source-added tests executed; 16 tests are intentionally disabled by GTest. - Five source regression suites on an isolated FE/BE pair: PASS (`test_match_format_parity`, `test_search_format_parity`, `test_search_inverted_index_profile`, `test_search_not_null_bitmap`, `test_search_segment_document_ids`). The temporary suite root contains byte-identical copies and uses the original golden data. - A separate clang-tidy 20 check of `vsearch.cpp` is blocked by an unmatched `NOLINTEND` in unchanged `be/src/core/types.h:578`; branch-4.2 has no `run-clang-tidy.sh`. - Complete PR diff English check: PASS (869 source files; 1,036 individually reviewed data occurrences; zero descriptive violations). - Committed merged-tree format preflight against `origin/branch-4.2`: PASS (1,000 changed files). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
