dgd-contributor commented on a change in pull request #33459:
URL: https://github.com/apache/spark/pull/33459#discussion_r678192485
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/NumberConverter.scala
##########
@@ -112,19 +127,19 @@ object NumberConverter {
var (negative, first) = if (n(0) == '-') (true, 1) else (false, 0)
// Copy the digits in the right side of the array
- val temp = new Array[Byte](64)
+ val temp = new Array[Byte](Math.max(n.length, 64))
var v: Long = -1
- if ((n.length == 65 && negative) || n.length <= 64) {
Review comment:
if n.length > 65, there are 2 cases can happen
case 1:
input string have format like: `valid Input String + invalid Character +
something` and the valid input string can converted without overflow. it will
ignore the suffix starts from the invalid character
example: `conv( ("a" * 4 + "?" * 60), 16, 10)` will have result as 43690 (as
in new test case)
case 2:
input string is a valid input string => it will over flow and return max
unsigned long value converted to toBase
example: `conv(
"aaaaaaa0aaaaaaa0aaaaaaa0aaaaaaa0aaaaaaa0aaaaaaa0aaaaaaa0aaaaaaa0aaaaaaa0", 16,
10)` will result 18446744073709551615 (this is from test case from SPARK-33428)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
