You can sort of fix this issue by rounding, but it just hides the issue and 
should not be used for financial applications:

var setPrecision:Function = function(number:Number, precision:int) {
 precision = Math.pow(10, precision);
 return Math.round(number * precision)/precision;
}

var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on
https://stackoverflow.com/questions/632802/how-to-deal-with-number-precision-in-actionscript
 
<https://stackoverflow.com/questions/632802/how-to-deal-with-number-precision-in-actionscript>

Erik

On Feb 2, 2018, at 8:39 AM, Erik J. Thomas <e...@linqto.com> wrote:

The Number data type, when including a decimal point is a binary floating point 
number and binary floating point math can result in rounding errors like yours. 

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680 
<https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#680> 

Adobe says this about Number:

"The Number data type uses the 64-bit double-precision format as specified by 
the IEEE Standard for Binary Floating-Point Arithmetic (IEEE-754) which means 
rounding errors can occur... The Number data type uses 52 bits to store the 
significand, with the result that numbers that require more than 52 bits to 
represent precisely, such as the fraction 1/3, are only approximations. If your 
application requires absolute precision with decimal numbers, you need to use 
software that implements decimal floating-point arithmetic as opposed to binary 
floating-point arithmetic."

Erik 

On Feb 2, 2018, at 7:22 AM, Σπύρος Αγγελόπουλος <aggelopou...@novusnet.gr> 
wrote:

Hi Everyone, 

Can anyone verify the results of calculation below ?



var n1:Number=18.9;

var n2:Number=100.0;              

var n3:Number=n1*n2;  //1889.9999999999998





is it bug ?



thanks 





spiros











Reply via email to