david-mollitor-db opened a new pull request, #58362:
URL: https://github.com/apache/spark/pull/58362
### What changes were proposed in this pull request?
`LikeSimplification` rewrites `col LIKE 'prefix%suffix'` into a length guard
plus
`StartsWith`/`EndsWith`:
```
Length(col) >= numChars(prefix) + numChars(suffix)
&& StartsWith(col, prefix) && EndsWith(col, suffix)
```
This PR changes the length guard to use `OctetLength` (byte length) instead
of `Length`
(character length), with the threshold expressed in bytes:
```
OctetLength(col) >= numBytes(prefix) + numBytes(suffix)
&& StartsWith(col, prefix) && EndsWith(col, suffix)
```
The guard exists only to reject strings too short to hold both the prefix
and the suffix
(e.g. `'a'` must not match `'a%a'`). `Length` is `numChars`, which is an
O(N) code-point
scan of the input string; `OctetLength` is the stored `numBytes`, which is
O(1).
### Why are the changes needed?
The character-length guard runs per row in the residual filter and performs
an O(N)
code-point count, whereas the byte length is already stored on `UTF8String`
and is O(1).
The two guards accept exactly the same set of strings, so the swap is
behavior-preserving.
`StartsWith` and `EndsWith` already pin the prefix and suffix at code-point
boundaries, so
any string that satisfies both anchors and is long enough in bytes to
contain them is also
long enough in code points, and vice versa. Concretely, for a string that
already passes
`StartsWith(prefix) && EndsWith(suffix)`:
```
numBytes(s) >= numBytes(prefix) + numBytes(suffix)
<=> numChars(s) >= numChars(prefix) + numChars(suffix)
```
### Does this PR introduce _any_ user-facing change?
No. The rewrite accepts the same rows as before; only the internal guard
expression changes
(`length` -> `octet_length`).
### How was this patch tested?
Updated and re-ran `LikeSimplificationSuite` (18/18 passing). For ASCII
prefixes/suffixes
the byte threshold equals the previous code-point threshold (`length(a) >=
6` becomes
`octet_length(a) >= 6`). The emoji case now guards on 8 bytes instead of 2
code points --
each of the two code points is a 4-byte UTF-8 sequence -- which accepts the
same strings.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
