[
https://issues.apache.org/jira/browse/SPARK-59371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59371:
-----------------------------------
Labels: pull-request-available (was: )
> LikeSimplification duplicates a non-cheap child in the `prefix%suffix`
> (startsAndEndsWith) rewrite
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-59371
> URL: https://issues.apache.org/jira/browse/SPARK-59371
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: David Mollitor
> Priority: Minor
> Labels: pull-request-available
>
> h2. Problem
> {{LikeSimplification}} rewrites a {{LIKE 'prefix%suffix'}} pattern – the
> {{startsAndEndsWith}} shape, e.g. {{'a%b'}} – into a length guard plus
> {{StartsWith}}
> plus {{{}EndsWith{}}}:
> {code:none}
> lengthGuard(input) && StartsWith(input, prefix) && EndsWith(input, postfix)
> {code}
> This references {{input}} three times. The single-{{{}Like{}}} branch of
> {{LikeSimplification.apply}} applies the rewrite unconditionally, unlike the
> {{{}LikeAll{}}}/{{{}NotLikeAll{}}}/{{{}LikeAny{}}}/{{{}NotLikeAny{}}}
> branches, which since SPARK-40228
> only fire when {{{}CollapseProject.isCheap(child){}}}. So a non-cheap child
> gets duplicated:
> * *Correctness* – a nondeterministic child (e.g. {{{}uuid(){}}},
> {{{}cast(rand() as string){}}}) is evaluated independently for each
> reference, so the three copies can produce different values and the rewritten
> predicate no longer matches the semantics of the original {{{}LIKE{}}}.
> Subexpression elimination does not help: it deliberately never deduplicates
> nondeterministic expressions.
> * *Performance* – an expensive deterministic child (e.g. {{{}sha2(col){}}})
> is written into
> the plan three times. Subexpression elimination usually collapses the repeats
> during codegen, but that is a physical optimization the logical plan should
> not rely on.
> The single-reference shapes ({{{}startsWith{}}}, {{{}endsWith{}}},
> {{{}contains{}}}, {{{}equalTo{}}})
> reference {{input}} once and are not affected.
> h2. Example
> {code:sql}
> -- uuid() is nondeterministic and returns a string
> SELECT * FROM t WHERE uuid() LIKE 'a%b'
> {code}
> is rewritten to roughly
> {code:sql}
> SELECT * FROM t
> WHERE length(uuid()) >= 2 AND startswith(uuid(), 'a') AND endswith(uuid(),
> 'b')
> {code}
> where each {{uuid()}} call draws a different value, so the length, prefix,
> and suffix
> checks are applied to three different strings – a result that can differ from
> the
> original single {{{}LIKE{}}}.
> h2. Fix
> Gate the {{startsAndEndsWith}} case on
> {{{}CollapseProject.isCheap(input){}}}. For a non-cheap child the pattern is
> left as a plain {{{}Like{}}}. The single-reference shapes evaluate the child
> exactly once, just like the original {{{}LIKE{}}}, so they stay enabled for
> any child.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
