airborne12 opened a new pull request, #70592: URL: https://github.com/apache/doris/pull/70592
### Background `MATCH_REGEXP` on a text index expands to the dictionary terms the expression matches. An expression without a fixed prefix walks the whole dictionary and runs the regular expression on every term. ### What problem does this PR solve? Issue Number: None Related PR: #70373 (the same prefilter for leading-wildcard globs) PR Type: Performance Problem Summary: Unanchored `MATCH_REGEXP` ran the regular expression on every dictionary term, although only terms holding its literal can match. This PR derives the literal every match must contain and skips the other terms before matching, cutting the expansion's CPU by 30-38% on V3 and 10-20% on SNII. ### Details - **Why it was slow:** every term went through the UTF conversion (V3) and a Hyperscan scan. - **Optimization:** the longest literal run of the expression's top-level concatenation becomes the pattern's required text, only for an expression without an enumeration prefix (CLucene tests the text before the prefix, so `^cach.` would otherwise walk past its range). - **Results:** 100M rows, caches off: V3 5.6-5.9 to 3.5-4.1 CPU-seconds, SNII 3.7-4.0 to 3.0-3.5; anchored and literal-free expressions unchanged; identical results. - **Applicability and cost:** unanchored regular expressions with a literal; one parse per pattern. ### Testing BE UT `TermPatternRegexpLiteral*`; regression `test_storage_format_snii_regexp_literal` (19 expressions, SNII against V3, expected output generated without this change); clang-tidy and the hygiene gate. ### Release note Unanchored MATCH_REGEXP queries skip the dictionary terms that cannot match before running the expression. ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: Claude Code (Claude Opus 5.5) - Generated: code, tests and this description - Human review: design and benchmark results reviewed by the author; code review pending 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
