davidm-db commented on code in PR #46206:
URL: https://github.com/apache/spark/pull/46206#discussion_r1579011842
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##########
@@ -214,6 +214,207 @@ public static int execICU(final UTF8String string, final
UTF8String substring,
}
}
+ public static class StringTrim {
+ public static UTF8String exec(
+ final UTF8String srcString,
+ final int collationId) {
+ CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if (collation.supportsBinaryEquality) {
+ return execBinary(srcString);
+ } else {
+ return execICU(srcString, collationId);
+ }
+ }
+ public static UTF8String exec(
+ final UTF8String srcString,
+ final UTF8String trimString,
+ final int collationId) {
+ CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if (collation.supportsBinaryEquality) {
+ return execBinary(srcString, trimString);
+ } else {
+ return execICU(srcString, trimString, collationId);
+ }
Review Comment:
You requested that we don't have `lcase` separately in the older PR:
https://github.com/apache/spark/pull/45749#discussion_r1546312983.
The logic is now embeded in `CollationAwareUTF8String.trimLeft` and
`CollationAwareUTF8String.trimRight`.
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##########
@@ -214,6 +214,207 @@ public static int execICU(final UTF8String string, final
UTF8String substring,
}
}
+ public static class StringTrim {
+ public static UTF8String exec(
+ final UTF8String srcString,
+ final int collationId) {
+ CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if (collation.supportsBinaryEquality) {
+ return execBinary(srcString);
+ } else {
+ return execICU(srcString, collationId);
+ }
+ }
+ public static UTF8String exec(
+ final UTF8String srcString,
+ final UTF8String trimString,
+ final int collationId) {
+ CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if (collation.supportsBinaryEquality) {
+ return execBinary(srcString, trimString);
+ } else {
+ return execICU(srcString, trimString, collationId);
+ }
Review Comment:
You requested that we don't have `lcase` separately implemented in the older
PR: https://github.com/apache/spark/pull/45749#discussion_r1546312983.
The logic is now embeded in `CollationAwareUTF8String.trimLeft` and
`CollationAwareUTF8String.trimRight`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
