Thanks for the response. Right as you were posting, I was posting my
own message of how I read up on the spec, and saw the same thing.
From what I have read, as you can't actually have the int portion be 9
999 999 999 999 999, because its limit is 9,007,199,254,740,992 , you
can really only trust the use of Number up to 15 digits of accuracy.
Let me know if I am wrong.
On 12/19/2010 4:33 AM, Gerry Beauregard wrote:
You're getting to the limits of the precision of a Number.
In ActionScript 3, the Number type is stored in binary, in 64-bit
double-precision IEEE floating point format:
http://en.wikipedia.org/wiki/Binary64
It uses 52 bits for the mantissa (fractional part) which gives it an effective
precision of 53 bits, equal to about 16 decimal digits of precision. The error
you're getting is in the 17th digit - no surprise at all.
-Gerry
On 2010-12-19 , at 09:55 , Anthony Pace wrote:
trace(Number('1000000009999992.2'));
//why does it output 1000000009999992.3
//I am assuming I am missing something pretty obvious
_______________________________________________
Flashcoders mailing list
[email protected]
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
_______________________________________________
Flashcoders mailing list
[email protected]
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
_______________________________________________
Flashcoders mailing list
[email protected]
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
