uros-db commented on code in PR #48521:
URL: https://github.com/apache/spark/pull/48521#discussion_r1806413319
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##########
@@ -500,9 +502,12 @@ public static UTF8String exec(final UTF8String srcString) {
}
public static UTF8String exec(
final UTF8String srcString,
- final UTF8String trimString,
+ UTF8String trimString,
final int collationId) {
CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if(collation.supportsSpaceTrimming) {
+ trimString = CollationFactory.applyTrimmingPolicy(trimString,
collationId);
Review Comment:
so trimming specifier absolutely shouldn't impact that behaviour at all.
however, what we do in fact want to impact is the case where the user doesn't
specify a space in the trimString:
- `trim(..., "xy")` // suppose: UTF8_BINARY_RTRIM
also please note that, as per `trim` function spec, we should absolutely not
remove any characters from the original input string (`srcString`) that the
user didn't specify in the `trimString` - because trimString is (by definition)
the set of characters that should be removed from the srcString
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
