mrhhsg opened a new pull request, #68501:
URL: https://github.com/apache/doris/pull/68501
### What problem does this PR solve?
Issue Number: None
Problem Summary:
With `enable_extended_regex = true`, a REGEXP/RLIKE pattern that RE2 rejects
(for example lookaround assertions such as `(?<=foo)bar` or `foo(?=123)`) is
compiled with Boost.Regex instead. That fallback only existed in the constant
pattern path built during `open()`. When the pattern comes from a column, or
from a constant that is only visible at execute time, `regexp_fn` and
`regexp_fn_scalar` compiled the pattern with RE2 only and failed the query
with `Invalid pattern: foo(?=123)` even though the same pattern worked as a
constant.
Reproduce:
```sql
SET enable_extended_regex = true;
CREATE TABLE regex_patterns (s VARCHAR(32), p VARCHAR(64))
DISTRIBUTED BY HASH(s) BUCKETS 1 PROPERTIES ("replication_num" = "1");
INSERT INTO regex_patterns VALUES ('foobar', '(?<=foo)bar'), ('foo123bar',
'foo(?=123)');
SELECT REGEXP('foobar', '(?<=foo)bar'); -- 1
SELECT s, p, s REGEXP p FROM regex_patterns; -- ERROR: Invalid
pattern: foo(?=123)
```
Fix: move the RE2-then-Boost compilation into a shared
`FunctionLikeBase::compile_regex` helper and a matching `regex_search`
helper, store `enable_extended_regex` in `LikeSearchState` next to
`enable_hyperscan_fallback`, and use the helpers in the constant path, the
per-batch `regexp_fn` path and the per-row `regexp_fn_scalar` path. After the
fix the column query returns `1` for both rows; when the session variable is
off, all three paths report the same
`Invalid regex expression: ... try setting enable_extended_regex=true` error
instead of the bare `Invalid pattern` message.
Two related hardenings in the same helpers: RE2 no longer logs every rejected
pattern to stderr (the error text is returned in the Status, and column
patterns are compiled once per row), and a `boost::regex_error` raised while
matching (Boost gives up on pathological patterns such as `(?=a)(a+)+b` once
its backtracking budget is exhausted) is converted into a non-OK Status
instead of escaping as a C++ exception, which would otherwise terminate the
process when the predicate is evaluated on a scanner thread.
### Release note
With `enable_extended_regex = true`, REGEXP/RLIKE now accepts the same
extended
patterns (for example lookaround assertions) when the pattern comes from a
column or another non-constant expression, not only when it is a constant. A
non-constant pattern that is still rejected now reports
`Invalid regex expression: ... try setting enable_extended_regex=true`
instead
of `Invalid pattern: ...`.
### Check List (For Author)
- Test:
- Unit Test: `FunctionLikeTest.regexp_extended_regex_all_pattern_modes`
covers lookaround patterns supplied as a plain column, as a constant
only visible at execute time and as a constant known at open, with the
session variable on and off, plus a pattern both engines reject and a
pattern Boost.Regex gives up on while matching.
- Regression test: `test_regexp_extended_nonconstant_pattern` compares
constant and column patterns, WHERE / NOT filters, the disabled-mode
error, a column pattern rejected by both engines and a column pattern
that fails while matching.
- Behavior changed: Yes. With `enable_extended_regex = true`, REGEXP/RLIKE
with a non-constant pattern now accepts the same patterns as a constant
pattern. The error for a rejected non-constant pattern now includes the RE2
error and the session-variable hint.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
