Re: [PR] [FLINK-36267][table] Fix SPLIT not preserving SMP characters when delimiter is empty [flink]

[via GitHub]

raminqaf commented on code in PR #28264:
URL: https://github.com/apache/flink/pull/28264#discussion_r3334321730


##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/CollectionFunctionsITCase.java:
##########
@@ -1813,14 +1813,16 @@ private Stream<TestSetSpec> splitTestCases() {
                                 ",123,123,",
                                 123,
                                 "12345",
-                                ",123,,,123,")
+                                ",123,,,123,",
+                                "123😊笑脸")

Review Comment:
   should we just harden the test with two SMP chars?
   ```suggestion
                                   "123😊👍🏽笑脸")
   ```



##########
flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/functions/scalar/SplitFunction.java:
##########
@@ -49,8 +49,12 @@ public SplitFunction(SpecializedFunction.SpecializedContext 
context) {
             if (delimiter.toString().isEmpty()) {
                 String str = string.toString();
                 List<StringData> res = new ArrayList<>();
-                for (int i = 0; i < str.length(); i++) {
-                    
res.add(StringData.fromString(String.valueOf(str.charAt(i))));
+                int i = 0;
+                while (i < str.length()) {
+                    int codePoint = str.codePointAt(i);
+                    int charCount = Character.charCount(codePoint);
+                    res.add(StringData.fromString(str.substring(i, i + 
charCount)));
+                    i += charCount;
                 }

Review Comment:
   nit: I think this reads better
   ```suggestion
                   while (i < str.length()) {
                       int codePoint = str.codePointAt(i);
                       int nextIndex = i + Character.charCount(codePoint);
                       res.add(StringData.fromString(str.substring(i, 
nextIndex)));
                       i = nextIndex;
   }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org