Mryange opened a new pull request, #70631: URL: https://github.com/apache/doris/pull/70631
### Background Segment scans remove predicates that zone maps prove always true. ANN Top-N falls back to brute-force vector scanning when residual predicates remain. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Problem Summary: Equality predicates were never recognized as always true, even when both segment bounds equal the comparison value. As a result, the implicit `__DORIS_DELETE_SIGN__ = 0` predicate could force ANN Top-N on MOW tables to scan every vector, including segments whose delete signs are all zero. Recognize non-negated constant equality predicates through the existing pruning path, allowing eligible segments to use the vector index with deletion visibility filtering enabled. ### Details - **Frequency:** Affects scans with an equality predicate that is true for the entire segment, including the default MOW delete-sign filter. - **Root cause:** `ComparisonPredicateBase::is_always_true()` handled range comparisons but not equality. - **Why existing tests missed it:** MOW ANN result tests can pass through brute-force fallback without proving index use. - **Fix:** Compare both bounds with the predicate value using the same equality semantics as row evaluation. Preserve NULL rejection and conservatively retain negated predicates. - **Other paths and branches:** Ordinary constant equality predicates benefit as well. Segments with mixed delete signs or other residual filters retain the existing fallback checks; delete bitmap filtering is unchanged. ### Testing Validated on the development machine with ASAN: - BE and FE builds passed. - `bash run-be-ut.sh --run --filter=BlockColumnPredicateTest.* -j48`: 20 tests passed, including four new tests for equality bounds, NULLs, negation, strings and NaN. - `bash run-regression-test.sh --run -d ann_index_p0 -s ann_index_on_mow`: passed. - clang-format 16, header hygiene and clang-tidy checks passed. On the same synthetic 10,000-row MOW table, the same Top-10 query with default `skip_delete_sign=false` changed from **0 ANN searches / 1 fallback / 10,000 rows read** to **1 ANN search / 0 fallbacks / 10 rows read**, with identical results. Additional result/Profile checks passed for constant ordinary equality, residual filters, UPSERT visibility, mixed tombstones and reinsertion. This validates correctness and index selection, not production latency or recall. ### Release note Fix unnecessary ANN Top-N brute-force fallback on MOW segments whose delete-sign equality is provably always true, while preserving deletion visibility filtering. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: Codex - Generated: Implementation, unit tests and PR description. - Human review: Author reviewed the equality pruning logic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
