[
https://issues.apache.org/jira/browse/SPARK-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205134#comment-15205134
]
Mark Hamstra commented on SPARK-13806:
--------------------------------------
Yes, there is the mostly orthogonal question about which rounding strategy
should be used -- see the comments in SPARK-8279. But, assuming that we are
adopting the ROUND_HALF_UP strategy, there is the problem with negative values
that this JIRA points out: When using ROUND_HALF_UP and scale == 0, -x.5 must
round to -(x+1), but Math.round will round it to -x.
In addition to this, the code gen for rounding of negative floating point
values with negative scales is broken.
All of this stems from Spark SQL's implementation of round() being untested
with negative values.
> SQL round() produces incorrect results for negative values
> ----------------------------------------------------------
>
> Key: SPARK-13806
> URL: https://issues.apache.org/jira/browse/SPARK-13806
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0, 1.6.1, 2.0.0
> Reporter: Mark Hamstra
>
> Round in catalyst/expressions/mathExpressions.scala appears to be untested
> with negative values, and it doesn't handle them correctly.
> There are at least two issues here:
> First, in the genCode for FloatType and DoubleType with _scale == 0, round()
> will not produce the same results as for the BigDecimal.ROUND_HALF_UP
> strategy used in all other cases. This is because Math.round is used for
> these _scale == 0 cases. For example, Math.round(-3.5) is -3, while
> BigDecimal.ROUND_HALF_UP at scale 0 for -3.5 is -4.
> Even after this bug is fixed with something like...
> {code}
> if (${ce.value} < 0) {
> ${ev.value} = -1 * Math.round(-1 * ${ce.value});
> } else {
> ${ev.value} = Math.round(${ce.value});
> }
> {code}
> ...which will allow an additional test like this to succeed in
> MathFunctionsSuite.scala:
> {code}
> checkEvaluation(Round(-3.5D, 0), -4.0D, EmptyRow)
> {code}
> ...there still appears to be a problem on at least the
> checkEvalutionWithUnsafeProjection path, where failures like this are
> produced:
> {code}
> Incorrect evaluation in unsafe mode: round(-3.141592653589793, -6), actual:
> [0,0], expected: [0,8000000000000000] (ExpressionEvalHelper.scala:145)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
