cloud-fan commented on code in PR #45592:
URL: https://github.com/apache/spark/pull/45592#discussion_r1537196013
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##########
@@ -65,26 +65,39 @@ public static class Collation {
public final BiFunction<UTF8String, UTF8String, Boolean> equalsFunction;
/**
- * Binary collation implies that UTF8Strings are considered equal only if
they are
- * byte for byte equal. All accent or case-insensitive collations are
considered non-binary.
+ * Support for Binary Equality implies that it is possible to check
equality on
+ * byte by byte level. This allows for the usage of binaryEquals call on
UTF8Strings
+ * which is more performant than calls to external ICU library.
*/
- public final boolean isBinaryCollation;
+ public final boolean supportsBinaryEquality;
+ /**
+ * Support for Binary Ordering implies that it is possible to check
equality and compare on
+ * byte by byte level. This allows for the usage of binaryEquals and
binaryCompare calls on
+ * UTF8Strings which is more performant than calls to external ICU
library. Support for
+ * Binary Ordering implies support for Binary Equality.
Review Comment:
oh we already did.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
