LuciferYang commented on code in PR #56569:
URL: https://github.com/apache/spark/pull/56569#discussion_r3432749082
##########
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java:
##########
@@ -1157,7 +1157,7 @@ public UTF8String reverse() {
int i = 0; // position in byte
while (i < numBytes) {
- int len = Math.min(numBytesForFirstByte(getByte(i)), numBytes);
+ int len = Math.min(numBytesForFirstByte(getByte(i)), numBytes - i);
Review Comment:
Thanks @cloud-fan, good catch. That sentence was inaccurate:
`codePointFrom`, `trimLeft`, and `trimRight` (the latter two via
`copyUTF8String`) all take the width from `numBytesForFirstByte` without
clamping to the remaining bytes, so they over-read on the same
truncated-trailing-sequence input. I've updated the description to drop that
claim and keep this PR scoped to `reverse()`, and filed SPARK-57520 to handle
the siblings (and the unclamped `copyUTF8String`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
