Kai Bächle created VELOCITY-996:
-----------------------------------
Summary: Precision loss when Long values are converted to
BigDecimal in VTL arithmetic
Key: VELOCITY-996
URL: https://issues.apache.org/jira/browse/VELOCITY-996
Project: Velocity
Issue Type: Bug
Components: Engine
Affects Versions: 2.4.1
Reporter: Kai Bächle
MathUtils.toBigDecimal(Number) appears to convert Number instances through
doubleValue() before creating a BigDecimal.
This loses precision for large integral values such as Long.MAX_VALUE. As a
result, VTL arithmetic and comparisons involving a Long and a BigDecimal can
produce unexpected results.
Minimal VTL example, without any Java context:
{code:java}
## Pure VTL test for MathUtils.toBigDecimal(Long) precision issue
#set($L = 9223372036854775807) ## Long.MAX_VALUE => Long
#set($BI = 9223372036854775808) ## one above Long.MAX_VALUE => BigInteger
## Create BigDecimal values without Java context:
#set($BD0 = ($BI + 0.0) - $BI) ## BigDecimal 0.0
#set($BDL = $BI - 1.0) ## BigDecimal 9223372036854775807.0
Long literal (L): $L ($L.class.simpleName)
BigInteger literal (BI): $BI ($BI.class.simpleName)
BigDecimal zero (BD0): $BD0 ($BD0.class.simpleName)
BigDecimal Long.MAX (BDL): $BDL ($BDL.class.simpleName)
-- OK: pure integer arithmetic --
#set($a = $L + 0)
L + 0 = $a ($a.class.simpleName)
#set($b = $L + 1)
L + 1 = $b ($b.class.simpleName)
-- Double path, precision loss is expected here --
#set($c = $L + 0.0)
L + 0.0 = $c ($c.class.simpleName)
-- BigDecimal path, unexpected precision loss --
#set($d = $L + $BD0)
L + BigDecimal(0.0) = $d ($d.class.name)
-- Comparison against exact BigDecimal Long.MAX_VALUE --
L == BigDecimal(Long.MAX) = #if($L == $BDL)true#{else}false#end
L > BigDecimal(Long.MAX) = #if($L > $BDL)true#{else}false#end{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
