MaxGekk commented on code in PR #48521:
URL: https://github.com/apache/spark/pull/48521#discussion_r1804879965
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##########
@@ -433,9 +439,12 @@ public static int execICU(final UTF8String string, final
UTF8String substring, f
}
public static class SubstringIndex {
- public static UTF8String exec(final UTF8String string, final UTF8String
delimiter,
+ public static UTF8String exec(final UTF8String string, UTF8String
delimiter,
final int count, final int collationId) {
CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if(collation.supportsSpaceTrimming){
Review Comment:
```suggestion
if (collation.supportsSpaceTrimming){
```
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala:
##########
@@ -796,6 +812,7 @@ case class MakeValidUTF8(input: Expression) extends
RuntimeReplaceable with Impl
}
+// TODO: fix
Review Comment:
please, add the `[WIP]` tag if the PR is not ready.
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##########
@@ -395,9 +401,12 @@ public static UTF8String execICU(final UTF8String src,
final UTF8String search,
}
public static class StringLocate {
- public static int exec(final UTF8String string, final UTF8String
substring, final int start,
+ public static int exec(final UTF8String string, UTF8String substring,
final int start,
final int collationId) {
CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if(collation.supportsSpaceTrimming){
Review Comment:
```suggestion
if (collation.supportsSpaceTrimming){
```
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##########
@@ -144,8 +149,12 @@ public static boolean execICU(final UTF8String l, final
UTF8String r,
}
public static class EndsWith {
- public static boolean exec(final UTF8String l, final UTF8String r, final
int collationId) {
+ public static boolean exec(UTF8String l, UTF8String r, final int
collationId) {
CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if(collation.supportsSpaceTrimming){
Review Comment:
```suggestion
if (collation.supportsSpaceTrimming){
```
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java:
##########
@@ -321,9 +326,12 @@ public static int execCollationAware(final UTF8String
word, final UTF8String set
}
public static class StringInstr {
- public static int exec(final UTF8String string, final UTF8String substring,
+ public static int exec(final UTF8String string, UTF8String substring,
final int collationId) {
CollationFactory.Collation collation =
CollationFactory.fetchCollation(collationId);
+ if(collation.supportsSpaceTrimming){
Review Comment:
```suggestion
if (collation.supportsSpaceTrimming){
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
