Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22227#discussion_r214562374
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -952,6 +952,11 @@ public static UTF8String concatWs(UTF8String
separator, UTF8String... inputs) {
}
public UTF8String[] split(UTF8String pattern, int limit) {
+ // Java String's split method supports "ignore empty string" behavior
when the limit is 0.
+ // To avoid this, we fall back to -1 when the limit is 0.
--- End diff --
Could you narrow down why we avoid the -1 case for others? thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
