jovanpavl-db commented on code in PR #48386:
URL: https://github.com/apache/spark/pull/48386#discussion_r1793114630
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##########
@@ -154,16 +154,24 @@ public static class Collation {
*/
public final boolean supportsLowercaseEquality;
+ /**
+ * Support for Space Trimming implies that that based on specifier (for
now only right trim)
+ * leading, trailing or both spaces are removed from the input string
before comparison.
+ */
+ public final boolean supportsSpaceTrimming;
+
public Collation(
String collationName,
String provider,
Collator collator,
Comparator<UTF8String> comparator,
String version,
ToLongFunction<UTF8String> hashFunction,
+ BiFunction<UTF8String, UTF8String, Boolean> equalsFunction,
boolean supportsBinaryEquality,
boolean supportsBinaryOrdering,
- boolean supportsLowercaseEquality) {
+ boolean supportsLowercaseEquality,
+ boolean supportsSpaceTrimming) {
Review Comment:
Yes, actually i thought about this and plan to address it in follow-up pr.
So yes, this name is not accurate any more. One could say, why just not then
set it on false for UTF8_BINARY collations using trimming? Well for many
expressions trim specifier is passthrough and we would like to have it done
together with UTF8_BINARY collation. Not to mention on many places
supportsBinaryEquality = isUtfBinary which is ok i guess because I guess it was
not planeed to extend UTF8_binary collaiton.
So simple solution for this would be to **rename supportsBinaryOrdering to
something like "isUtf8BinaryType"**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
