vitaliili-db commented on code in PR #37631:
URL: https://github.com/apache/spark/pull/37631#discussion_r953973254
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala:
##########
@@ -527,7 +527,7 @@ case class StringSplit(str: Expression, regex: Expression,
limit: Expression)
override def second: Expression = regex
override def third: Expression = limit
- def this(exp: Expression, regex: Expression) = this(exp, regex, Literal(-1));
+ def this(exp: Expression, regex: Expression) = this(exp, regex, Literal(0))
Review Comment:
Yes, you are correct. This should not change. Reverted.
##########
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java:
##########
@@ -1000,6 +1000,17 @@ public static UTF8String concatWs(UTF8String separator,
UTF8String... inputs) {
}
public UTF8String[] split(UTF8String pattern, int limit) {
+ // This is a special case for converting string into array of symbols
without a trailing empty
+ // string. E.g. `"hello".split("", 0) => ["h", "e", "l", "l", "o"].
+ // Note that negative limit will preserve a trailing empty string.
+ if (limit == 0 && numBytes() != 0 && pattern.numBytes() == 0) {
+ byte[] input = getBytes();
+ UTF8String[] result = new UTF8String[numBytes];
+ for (int i = 0; i < numBytes; i++) {
Review Comment:
Great catch, fixed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
