david-mollitor-db opened a new pull request, #59223:
URL: https://github.com/apache/spark/pull/59223
### What changes were proposed in this pull request?
When the pattern of `RLIKE` or `LIKE` is foldable, the generated code
compiles the `Pattern` once into mutable state, but still calls
`pattern.matcher(input)` on every row. This PR builds the `Matcher` once in the
mutable state's init code and resets it per row instead:
```java
// before
value = patternRLike.matcher(input.toString()).find(0);
// after; init code: matcherRLike = Pattern.compile("<regex>",
<flags>).matcher("");
value = matcherRLike.reset(input.toString()).find(0);
```
`LIKE` gets the same change, using `matches()`. It applies to patterns that
`LikeSimplification` does not rewrite into `StartsWith` / `EndsWith` /
`Contains`, such as `a%b%c`.
- The Matcher replaces the Pattern field (a Matcher holds its Pattern), so
the number of mutable states does not change.
- Codegen mutable state belongs to one generated-object instance, so the
Matcher is never shared between threads. The child is evaluated before `reset`,
and each match finishes before the next row starts.
- Not changed: the interpreted path (`RLike` / `Like` are not `stateful`, so
a Matcher cached on the expression would be shared), non-foldable patterns,
`LIKE ANY` / `LIKE ALL`, and the per-row `UTF8String.toString()`.
### Why are the changes needed?
Each `pattern.matcher(...)` call allocates a `Matcher` along with its
`groups` and `locals` `int[]` and its `localsPos` `IntHashSet[]`. The JIT
cannot scalar-replace these objects because the Matcher escapes into the regex
engine's recursive `Node.match` calls. In allocation profiles of a filter with
a token-boundary `RLIKE` (`(?<![a-z0-9])word(?![a-z0-9])`), `Matcher.<init>` <-
`Pattern.matcher` <- generated `processNext` accounted for 2-5% of allocated
bytes.
Local benchmark: JDK 17, `local[1]`, 5M cached rows of short strings such as
`item 123 sword tail`, each case run as `selectExpr(<predicate>).noop()`, with
before and after runs interleaved. Values are ns per row:
| Case | Before (3 runs) | After (2 runs) | Change |
|---|---|---|---|
| control: `s LIKE '%tail'` (rewritten to `EndsWith`, not affected) | 50.4 -
52.9 | 50.9 - 51.6 | none |
| `s RLIKE '(?<![a-z0-9])word(?![a-z0-9])'` | 266.4 - 268.5 | 251.0 - 253.4
| -6% |
| `s RLIKE '^item 1'` | 89.8 | 85.3 | -5% |
| `s LIKE 'item%word%tail'` | 240.7 - 248.1 | 233.2 - 236.0 | -4% |
One more "after" run, taken right after a build, was slower on every case,
the control included, so it is left out.
Under JFR, sampled allocation for the whole benchmark run fell from 67.0 GB
to 27.3 GB. In the before run, `int[]`, `java.util.regex.Matcher` and
`IntHashSet[]` together made up about 70% of allocation pressure. None of them
appear among the top allocating classes in the after run.
This benchmark is not part of the PR. I can add cases to
`StringFunctionsBenchmark` if reviewers want them.
### Does this PR introduce _any_ user-facing change?
No. Only performance changes.
### How was this patch tested?
- New test in `RegexpExpressionsSuite`: it sends a sequence of rows through
one generated projection that holds two `RLIKE` expressions and one `LIKE`. The
rows include matches, non-matches, a null, an empty string, and a long input
followed by a short one, and the test checks that no result depends on the rows
before it.
- Existing `RegexpExpressionsSuite`, `CollationRegexpExpressionsSuite` and
`CollationSQLRegexpSuite` pass, as do the `regexp-functions.sql` and
`like-*.sql` golden files in `SQLQueryTestSuite`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Isaac (Claude Code, Claude Opus 5.5)
This pull request and its description were written by Isaac.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
