uros-db commented on code in PR #45978:
URL: https://github.com/apache/spark/pull/45978#discussion_r1561054469

@@ -172,19 +183,31 @@ public Collation(
-   * Auxiliary methods for collation aware string operations.
+   * Returns a StringSearch object for the given pattern and target strings, 
under collation
+   * rules corresponding to the given collationId. The external ICU library 
StringSearch object can
+   * be used to find occurrences of the pattern in the target string, while 
respecting collation.
   public static StringSearch getStringSearch(
-      final UTF8String left,
-      final UTF8String right,
+      final UTF8String targetUTF8String,
+      final UTF8String patternUTF8String,
       final int collationId) {
-    String pattern = right.toString();
-    CharacterIterator target = new StringCharacterIterator(left.toString());
+    String pattern = patternUTF8String.toString();
+    CharacterIterator target = new 
     Collator collator = CollationFactory.fetchCollation(collationId).collator;
     return new StringSearch(pattern, target, (RuleBasedCollator) collator);
+  /**
+   * Returns a collation-unaware StringSearch object for the given pattern and 
target strings.
+   * While this object does not respect collation, it can be used to find 
occurrences of the pattern
+   * in the target string for UTF8_BINARY or UTF8_BINARY_LCASE (if arguments 
are lowercased).
+   */
+  public static StringSearch getStringSearch(

Review Comment:
   That part becomes a little complicated with expressions that need to 
preserve the original string value, consider StringReplace: 
`'BAbaBA'.replace('ab', 'XX')` with UTF8_BINARY_LCASE should return 'BXXaBA' 
(not 'bxxaba')
   so .toLowerCase version is used for matching, and original version 
(non-Lowercase) is used for replacement
   @miland-db's currently 2 open PRs use this variant of StringSearch, although 
it's not clear if that's definitely a good approach. That said, I do think we 
can definitely figure out ways to do this in so that we rely on `UTF8String` 
functions rather than `StringSearch` for UTF8_BINARY_LCASE, but we're yet to 
figure out if that's really more efficient (or even easier to implement) - for 
example, Milan noticed that the `UTF8String` implementation of "translate" 
actually does ".toString()" followed by a bunch of ".substring" calls, while 
StringSearch uses a `CharacterIterator` which should probably be more efficient
   we may consider benchmarking this, but in terms of this PR @cloud-fan I 
think we could agree on either:
   1. keeping this version of `getStringSearch` here until we're sure that 
we're not gonna need it for any expression (we could remove it once we've 
swooped over all expressions)
   2. removing this version of `getStringSearch` here until we're sure that we 
need it somewhere (we could add it back once there's a guaranteed need for it)

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to