xumingming opened a new pull request, #60057: URL: https://github.com/apache/spark/pull/60057
### What changes were proposed in this pull request? Add an optimizer rule, disabled by default, that rewrites an eligible constant `RLIKE` into a balanced OR of `Contains` predicates. The input must be `UTF8_BINARY`. After redundant leading and trailing `.*` and plain outer groups are removed, every alternative must be a non-empty literal. Anchors, escapes, empty alternatives, other metacharacters, non-binary collations, and a non-deterministic input duplicated across alternatives stay on the regex path. The switch is `spark.sql.optimizer.rlikeSimplification.enabled` (default false, `NOT_APPLICABLE`, version 4.4.0). JIRA: https://issues.apache.org/jira/browse/SPARK-60844 ### Why are the changes needed? `RLIKE` uses `Matcher.find`. A pattern such as `.*(foo|bar).*` is a containment test, but Java regex still walks the alternation on every row. `LikeSimplification` already rewrites `LIKE '%foo%'` to `Contains`. There is no counterpart for this `RLIKE` shape. ### Does this PR introduce _any_ user-facing change? Yes, behind a default-off config. With the config left false, `RLIKE` planning and results are unchanged. With it set to true, an eligible literal alternation is evaluated as string containment instead of a regular expression. Patterns that fail the guards above are unchanged. ### How was this patch tested? Added `RLikeSimplificationSuite`, plus enabled-mode coverage in `StringFunctionsSuite` (including `CODEGEN_ONLY`), `CollationSQLRegexpSuite` (UTF8_LCASE dotless i stays on `RLIKE`), and `DataSourceV2DataFrameSuite` (disabled mode pushes only `IsNotNull`; enabled mode pushes the `Contains` disjunction). `RLikeSimplificationSuite` passed 10/10. `RLikeSimplificationBenchmark` compares the config off and on. Local run, OpenJDK 17.0.18, Apple M4 Pro, 1000 nonmatching rows, 3 iterations. Values are a padding of `x` plus the row id, so nothing matches and the OR-tree cannot short-circuit. Relative is the benchmark Relative column for enabled versus disabled best time. | Case | Disabled best (ms) | Enabled best (ms) | Relative | | --- | ---: | ---: | ---: | | Short input (32), 4 alternatives | 19 | 6 | 3.0X | | Long input (256), 4 alternatives | 455 | 5 | 83.6X | | Short input (32), 32 alternatives | 59 | 5 | 12.2X | | Long input (256), 32 alternatives | 2747 | 8 | 337.5X | | `lower(upper(value))`, 32 alternatives | 59 | 6 | 10.0X | This is a local microbenchmark, not a GitHub Actions result. The timed query is `select(rlike(...)).noop()` and includes planning. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Pi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
