SEPURI-SAI-KRISHNA commented on PR #57931: URL: https://github.com/apache/spark/pull/57931#issuecomment-5603184799
@stevomitric good point, and it pushed me to a better fix than the one you were looking at. `numChars` memoizes per instance, but the first call is an O(n) scan and each row brings a fresh `UTF8String`, so that cost landed on every call to spare a single extreme position. Working out when the truncation actually happens: `substringSQL` derives its end offset from `start + Int.MaxValue` where `start = numChars + pos`, so the tail reaches the end of the input whenever `pos + Int.MaxValue >= 0`, that is for every position down to `Int.MinValue + 1`. `Int.MinValue` itself is the only value that falls short, by exactly one character. So the clamp floor just needs to be `Int.MinValue + 1` rather than `Int.MinValue`. No input length, no extra branch, and the binary overload drops its length lookup too. The tests are unchanged and still cover the zero-length case, `Int.MinValue + 1`, multi-byte input and empty input. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
