stevomitric commented on code in PR #58484:
URL: https://github.com/apache/spark/pull/58484#discussion_r3970629684
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala:
##########
@@ -863,6 +851,43 @@ object LikeSimplification extends Rule[LogicalPlan] with
PredicateHelper {
}
}
+ // For a leading-literal pattern that `simplifyLike` leaves as a full `Like`
(e.g. 'a%b%'),
+ // derive the necessary condition `StartsWith(input, <leading literal>)` and
keep the `Like`
+ // as the exact residual: `StartsWith(input, prefix) && (input LIKE
pattern)`. `StartsWith` is
+ // placed first so the cheap check short-circuits the regex, and it can be
pushed to data
+ // sources (e.g. Parquet prunes row groups on `StringStartsWith`) while the
`Like` re-checks
+ // the match exactly.
+ //
+ // Restricted to collations with binary equality: only then do the `Like`
regex match and
+ // `StartsWith` agree byte-for-byte, so `StartsWith(prefix)` is a sound
necessary condition of
+ // the `Like` (under e.g. UTF8_LCASE the two matchers can disagree, risking
a false negative),
+ // and only then does `StringStartsWith` push down. The residual `Like` is
tagged so the rule
+ // stays idempotent under the fixed-point batch.
+ private def derivePrefixStartsWith(
+ input: Expression,
+ pattern: String,
+ escapeChar: Char,
+ like: Expression): Option[Expression] = {
+ val binaryCollation = input.dataType match {
+ case st: StringType => st.supportsBinaryEquality
+ case _ => false
+ }
+ if (!binaryCollation || !CollapseProject.isCheap(input) ||
pattern.contains(escapeChar) ||
+ like.containsTag(LIKE_PREFIX_GUARDED)) {
+ None
+ } else {
+ val prefix = pattern.takeWhile(c => c != '%' && c != '_')
+ if (prefix.isEmpty || prefix.length == pattern.length) {
Review Comment:
this bails whenever the escape char appears anywhere in the pattern, but the
leading literal can be escape-free while the escape only appears later, e.g.
'ab%c\%d%', whose prefix ab is clean.
we could have something like:
```scala
if (!binaryCollation || like.containsTag(LIKE_PREFIX_GUARDED)) {
None
} else {
val prefix = pattern.takeWhile(c => c != '%' && c != '_')
if (prefix.isEmpty || prefix.length == pattern.length ||
prefix.contains(escapeChar)) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
