srowen commented on a change in pull request #28750:
URL: https://github.com/apache/spark/pull/28750#discussion_r436668197
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringUtils.scala
##########
@@ -122,8 +122,8 @@ object StringUtils extends Logging {
val available = maxLength - length
val stringToAppend = if (available >= sLen) s else s.substring(0,
available)
strings.append(stringToAppend)
+ length += sLen
}
- length += sLen
Review comment:
Nitpicking, but:
Why not only update length inside the `if (!atLimit)` block as originally
shown?
And only add `stringToAppend.length()` there, if the point of `Math.min()`
is to avoid overflow if adding a massive string where only part was added.
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringUtils.scala
##########
@@ -122,8 +122,8 @@ object StringUtils extends Logging {
val available = maxLength - length
val stringToAppend = if (available >= sLen) s else s.substring(0,
available)
strings.append(stringToAppend)
+ length += sLen
}
- length += sLen
Review comment:
Oh I see. If overflow is an issue, then make it a long? Capping it at
this constant would then seem to potentially underreport the actual appended
total.
Maybe that creates other problems, not sure.
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringUtils.scala
##########
@@ -122,8 +122,8 @@ object StringUtils extends Logging {
val available = maxLength - length
val stringToAppend = if (available >= sLen) s else s.substring(0,
available)
strings.append(stringToAppend)
+ length += sLen
}
- length += sLen
Review comment:
I'm not sure you can overflow a long - if 2^31-1 strings (max length of
the array tracking them) are added, of length 2^31-1, you still don't overflow.
It's probably not worth debating so this is fine but I think a simpler
change was also fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
