[
https://issues.apache.org/jira/browse/SPARK-59968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59968:
-----------------------------------
Labels: pull-request-available (was: )
> Reuse the regex Matcher across rows in LIKE and RLIKE codegen
> -------------------------------------------------------------
>
> Key: SPARK-59968
> URL: https://issues.apache.org/jira/browse/SPARK-59968
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: David Mollitor
> Priority: Minor
> Labels: pull-request-available
>
> h3. Problem
> When the pattern of {{RLIKE}} or {{LIKE}} is foldable, the generated code
> compiles the {{java.util.regex.Pattern}} once and keeps it in mutable state.
> However, it still calls {{pattern.matcher(input)}} on every row:
> {code:java}
> value = patternRLike.matcher(input.toString()).find(0);
> {code}
> Each call allocates a new {{java.util.regex.Matcher}} along with its internal
> {{{}groups{}}}, {{locals}} and {{localsPos}} arrays. The JIT cannot
> scalar-replace this allocation because the Matcher escapes into the regex
> engine's recursive {{Node.match}} graph. Allocation profiles of a filter
> using a token-boundary {{RLIKE}} show {{Matcher.<init>}} <-
> {{Pattern.matcher}} <- generated {{processNext}} at roughly 2-5% of allocated
> bytes.
> {{LIKE}} has the same shape (using {{{}matches(){}}}) for patterns that
> {{LikeSimplification}} does not rewrite into {{StartsWith}} / {{EndsWith}} /
> {{{}Contains{}}}, such as {{{}a%b%c{}}}.
> h3. Proposal
> In the foldable branch of {{RLike.doGenCode}} and {{{}Like.doGenCode{}}},
> build the {{Matcher}} once in the mutable state's init code, and reset it per
> row instead of allocating a new one:
> {code:java}
> // init
> matcherRLike = java.util.regex.Pattern.compile("<regex>",
> <flags>).matcher("");
> // per row
> value = matcherRLike.reset(input.toString()).find(0);
> {code}
> * The Matcher replaces the Pattern field (the Matcher holds its Pattern), so
> the number of mutable states does not change.
> * Codegen mutable state is per generated-object instance, so the Matcher is
> never shared across threads.
> * Results are unchanged; {{find(0)}} / {{matches()}} keep the same semantics
> as the interpreted path.
> Out of scope: the interpreted path (RLike/Like are not {{{}stateful{}}}),
> non-foldable patterns, LIKE ANY / LIKE ALL, and the per-row
> {{UTF8String.toString()}} conversion.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
