[
https://issues.apache.org/jira/browse/SPARK-30759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033889#comment-17033889
]
Dongjoon Hyun edited comment on SPARK-30759 at 2/10/20 7:48 PM:
----------------------------------------------------------------
Hi, [~maxgekk]. As you wrote on the PR, this is a behavior since 1.0.0. So, I
convert this into an improvement JIRA. That will be easier to understand this
issue impact and contribution.
If you PR doesn't have another improvement aspect (Literal to Foldable), the
scope might be considered differently.
was (Author: dongjoon):
Hi, [~maxgekk]. As you wrote on the PR, this is a behavior since 1.0.0. So, I
convert this into an improvement JIRA. That will be easier to understand this
issue impact and contribution.
> The cache in StringRegexExpression is not initialized for foldable patterns
> ---------------------------------------------------------------------------
>
> Key: SPARK-30759
> URL: https://issues.apache.org/jira/browse/SPARK-30759
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: Maxim Gekk
> Priority: Minor
> Attachments: Screen Shot 2020-02-08 at 22.45.50.png
>
>
> In the case of foldable patterns, the cache in StringRegexExpression should
> be evaluated once but in fact it is compiled every time. Here is the example:
> {code:sql}
> SELECT '%SystemDrive%\Users\John' _FUNC_ '%SystemDrive%\\Users.*';
> {code}
> the code
> https://github.com/apache/spark/blob/8aebc80e0e67bcb1aa300b8c8b1a209159237632/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala#L45-L48:
> {code:scala}
> // try cache the pattern for Literal
> private lazy val cache: Pattern = pattern match {
> case Literal(value: String, StringType) => compile(value)
> case _ => null
> }
> {code}
> The attached screenshot shows that foldable expression doesn't fall to the
> first case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
