Doris-Breakwater commented on issue #70588: URL: https://github.com/apache/doris/issues/70588#issuecomment-6892847414
Source-based triage: the reported inconsistency is supported by the code, and declining an incapable index is a reasonable fix direction for plain `MATCH_PHRASE`. There is one important correction to the reproduction, and the change needs regression coverage before treating all phrase variants as equivalent. **Verified in source** - In the local `4.1.4` tag, `FunctionMatchBase::evaluate_inverted_index` returns `INDEX_INVALID_PARAMETERS` for a fulltext reader without phrase support; a missing iterator instead returns OK without a result. The same rejection exists at the cited master revision, although master correctly checks the reader selected for the query/analyzer. Preserve that reader selection in any patch. See [4.1.4 evaluation](https://github.com/apache/doris/blob/4.1.4/be/src/exprs/function/match.cpp#L49-L109) and [master reader selection](https://github.com/apache/doris/blob/42b148bac33/be/src/exprs/function/match.cpp#L319-L352). - The fallback contract supports the proposed approach: `VExpr::_evaluate_inverted_index` records an exact result and marks the column as index-evaluated only when a result object has been populated. With no result, `VMatchPredicate` can execute the row function. Here `InvertedIndexResultBitmap::is_empty()` means **both bitmap pointers are null**, not “zero matching rows.” Return with the output untouched; do not populate a zero-cardinality bitmap, which would mean an exact no-match result. See [result registration](https://github.com/apache/doris/blob/42b148bac33/be/src/exprs/vexpr.cpp#L1056-L1079) and [bitmap representation](https://github.com/apache/doris/blob/42b148bac33/be/src/storage/index/inverted/inverted_index_reader.h#L215-L220). - Row execution calls `FunctionMatchBase::check`, so `enable_match_without_inverted_index=false` still rejects the slow path when it is actually executed. No broader suppression of `INDEX_INVALID_PARAMETERS` is needed. The normal segment downgrade handler does not classify that error as a fallback condition. **Reproduction correction** For a newly created index with `"parser" = "unicode"`, omitting `support_phrase` does **not** reliably create the intended non-positional index: the `4.1.4` FE `Index` constructor inserts `"support_phrase" = "true"` when a parser is specified and the property is absent. See [index defaults](https://github.com/apache/doris/blob/4.1.4/fe/fe-core/src/main/java/org/apache/doris/catalog/Index.java#L70-L100). Please use this explicitly in the reproducer and regression test: ```sql INDEX idx_msg (msg) USING INVERTED PROPERTIES ("parser" = "unicode", "support_phrase" = "false") ``` This does not invalidate the reported behavior for an index that actually lacks positions, but the supplied DDL alone does not establish that condition. **Suggested next steps for the PR** 1. Limit the behavioral change to the known lack-of-phrase-capability branch, leaving the bitmap untouched and retaining existing validation for other errors. The shared guard also covers `MATCH_PHRASE_PREFIX` and `MATCH_PHRASE_EDGE`; either explicitly cover those functions or scope the change accordingly. 2. Add result-based regression tests with explicit `support_phrase=false`: direct filtering versus row evaluation, AND/OR/NOT, a projection or join case, NULL and zero-match inputs, and mixed indexed/unindexed segments. Verify the two-row example returns only `id=1`, and verify the existing slow-path error with fallback disabled. Retain a positive test proving a compatible positional index is still used, including multiple indexes/analyzers. 3. Define the phrase-options boundary. The fulltext row path does not interpret slop suffixes as the index phrase path does; at the cited master revision, slop stripping in row execution is restricted to the keyword case. Thus the proposed fallback does not establish general semantic parity for queries such as `'a b ~2'`. Cover/document the supported behavior rather than claiming equivalence for all phrase syntax. 4. Keep the `MATCH_ALL` prefilter as a separate optimization. For a plain exact phrase, matching all analyzed terms can provide a candidate superset, but the phrase must remain for row verification. Do not register that candidate bitmap as an exact phrase result or generalize the rewrite to prefix/edge/options without proving it cannot discard valid rows. **Remaining evidence / limits** I inspected source, including `42b148bac33`, but did not execute the SQL on a running cluster. Please attach the corrected reproducer's actual outputs, `SHOW CREATE TABLE`, exact FE/BE build revisions, both `EXPLAIN VERBOSE` plans, the relevant session settings, and the failing query ID with its BE error excerpt. These will establish which path the production queries actually took and whether their segments lack an index or contain an index without positions. An unrelated OR is not a reliable workaround: compound-expression error handling also differs between `4.1.4` and the cited master revision, and the optimizer may still attempt index evaluation. The source evidence supports proceeding with a focused PR; it does not yet prove every reported production plan or segment configuration. Breakwater-GitHub-Analysis-Slot: slot_d7943bc41e46 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
