yaooqinn commented on a change in pull request #26640: [SPARK-30000][SQL] Trim
the string when cast string type to decimals
URL: https://github.com/apache/spark/pull/26640#discussion_r349628237
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
##########
@@ -632,7 +632,7 @@ abstract class CastBase extends UnaryExpression with
TimeZoneAwareExpression wit
private[this] def castToDecimal(from: DataType, target: DecimalType): Any =>
Any = from match {
case StringType =>
buildCast[UTF8String](_, s => try {
- changePrecision(Decimal(new JavaBigDecimal(s.toString)), target)
+ changePrecision(Decimal(new JavaBigDecimal(s.toString.trim)), target)
Review comment:
`s.toString.trim ` is better
```
[info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.1
[info] Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
[info] Cast String to Integral: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
[info]
------------------------------------------------------------------------------------------------------------------------
[info] cast(trim(str) as decimal) as c_decimal - with 1 spaces
3473 5428 491 2.4 424.0 1.0X
[info] cast(trim(str) as decimal) as c_decimal - with 3 spaces
3409 4163 706 2.4 416.1 1.0X
[info] cast(trim(str) as decimal) as c_decimal - with 5 spaces
6180 7422 795 1.3 754.3 0.6X
[info] cast(str as decimal) as c_decimal - with 1 spaces 3684
6684 1239 2.2 449.7 0.9X
[info] cast(str as decimal) as c_decimal - with 3 spaces 3516
4815 1443 2.3 429.2 1.0X
[info] cast(str as decimal) as c_decimal - with 5 spaces 6344
7338 755 1.3 774.5 0.5X
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
