xiaoxuandev opened a new pull request, #54665: URL: https://github.com/apache/spark/pull/54665
### What changes were proposed in this pull request? Fix escapeLikeRegex and LikeSimplification to correctly handle supplementary Unicode characters (code points above U+FFFF, e.g. emojis), which are stored as surrogate pairs in Java's UTF-16 String representation. ### Why are the changes needed? Two bugs when LIKE patterns contain supplementary characters: 1. escapeLikeRegex iterated by Char (UTF-16 code unit), splitting surrogate pairs into two separate regex quotations. Fixed by iterating with codePoints(). 2. LikeSimplification used String.length (UTF-16 code unit count) for the Length guard, but Spark's Length() returns code point count. Fixed by using codePointCount(). ### Does this PR introduce _any_ user-facing change? Yes. LIKE patterns with emojis and other supplementary characters now match correctly. ### How was this patch tested? Added tests in StringUtilsSuite, RegexpExpressionsSuite, and LikeSimplificationSuite. Verified test fails without the fix. Build and scalastyle pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
