[
https://issues.apache.org/jira/browse/SPARK-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Hamstra updated SPARK-13806:
---------------------------------
Description:
Round in catalyst/expressions/mathExpressions.scala appears to be untested with
negative values, and it doesn't handle them correctly.
There are at least two issues here:
First, in the genCode for FloatType and DoubleType with _scale == 0, round()
will not produce the same results as for the BigDecimal.ROUND_HALF_UP strategy
used in all other cases. This is because Math.round is used for these _scale
== 0 cases. For example, Math.round(-3.5) is -3, while
BigDecimal.ROUND_HALF_UP at scale 0 for -3.5 is -4.
Even after this bug is fixed with something like...
{code}
if (${ce.value} < 0) {
${ev.value} = -1 * Math.round(-1 * ${ce.value});
} else {
${ev.value} = Math.round(${ce.value});
}
{code}
...which will allow an additional test like this to succeed in
MathFunctionsSuite.scala:
{code}
checkEvaluation(Round(-3.5D, 0), -4.0D)
{code}
...there still appears to be a problem on at least the
checkEvalutionWithUnsafeProjection path, where failures like this are produced:
{code}
Incorrect evaluation in unsafe mode: round(-3.141592653589793, -6), actual:
[0,0], expected: [0,8000000000000000] (ExpressionEvalHelper.scala:145)
{code}
was:
Round in catalyst/expressions/mathExpressions.scala appears to be untested with
negative values, and it doesn't handle them correctly.
There are at least two issues here:
1) In the genCode for FloatType and DoubleType with _scale == 0, round() will
not produce the same results as for the BigDecimal.ROUND_HALF_UP strategy used
in all other cases. This is because Math.round is used for these _scale == 0
cases. For example, Math.round(-3.5) is -3, while BigDecimal.ROUND_HALF_UP at
scale 0 for -3.5 is -4.
Even after this bug is fixed with something like...
{code}
if (${ce.value} < 0) {
${ev.value} = -1 * Math.round(-1 * ${ce.value});
} else {
${ev.value} = Math.round(${ce.value});
}
{code}
...which will allow an additional test like this to succeed in
MathFunctionsSuite.scala:
{code}
checkEvaluation(Round(-3.5D, 0), -4.0D)
{code}
...there still appears to be a problem on at least the
checkEvalutionWithUnsafeProjection path, where failures like this are produced:
{code}
Incorrect evaluation in unsafe mode: round(-3.141592653589793, -6), actual:
[0,0], expected: [0,8000000000000000] (ExpressionEvalHelper.scala:145)
{code}
> SQL round() produces incorrect results for negative values
> ----------------------------------------------------------
>
> Key: SPARK-13806
> URL: https://issues.apache.org/jira/browse/SPARK-13806
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0, 1.6.1, 2.0.0
> Reporter: Mark Hamstra
>
> Round in catalyst/expressions/mathExpressions.scala appears to be untested
> with negative values, and it doesn't handle them correctly.
> There are at least two issues here:
> First, in the genCode for FloatType and DoubleType with _scale == 0, round()
> will not produce the same results as for the BigDecimal.ROUND_HALF_UP
> strategy used in all other cases. This is because Math.round is used for
> these _scale == 0 cases. For example, Math.round(-3.5) is -3, while
> BigDecimal.ROUND_HALF_UP at scale 0 for -3.5 is -4.
> Even after this bug is fixed with something like...
> {code}
> if (${ce.value} < 0) {
> ${ev.value} = -1 * Math.round(-1 * ${ce.value});
> } else {
> ${ev.value} = Math.round(${ce.value});
> }
> {code}
> ...which will allow an additional test like this to succeed in
> MathFunctionsSuite.scala:
> {code}
> checkEvaluation(Round(-3.5D, 0), -4.0D)
> {code}
> ...there still appears to be a problem on at least the
> checkEvalutionWithUnsafeProjection path, where failures like this are
> produced:
> {code}
> Incorrect evaluation in unsafe mode: round(-3.141592653589793, -6), actual:
> [0,0], expected: [0,8000000000000000] (ExpressionEvalHelper.scala:145)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
