RE: [flexcoders] In actionscript, converting from string to number loses precision

Thu, 09 Mar 2006 09:19:20 -0800

This actually exists in Flash 8 as well.  And I don't think it's a case of trace( ) not calling parseFloat, I get the same thing when I output it to a textfield.

And it gets better:

stop();
var s:String = "952.85";
var i:Number = parseFloat(s);

// txtResultX are dynamic textbox on stage
txtResult1 = Math.floor(952.85 / 0.05);   // 19057, correct
txtResult2 = Math.floor(i / 0.05);             // 19056, wrong
txtResult3 = Math.floor(i * ( 1 / 0.05));   // 19057, correct

- boon

Gordon Smith <[EMAIL PROTECTED]> wrote:
This is a bug. Can you please file it?
 
Apparently when the AS3 compiler compiles trace(i - 952.86), it doesn't convert the 952.86 to the exact same Number that the player does when it executes parseFloat() on "952.86". They should produce identical Numbers, and the different should be exactly 0.
 
You should be aware, however, that the Number they produce will be very close but not exactly equal to decimal 952.86. The reason is that Number stores values as binary fractions, not decimal fractions. A decimal fraction like 0.375 has a terminating binary representation (0.011 =0 x 1/2 + 1 x 1/4 + 1 x 1/8) but a decimal fraction like 952.86 has a non-terminating binary representation that would require an infinite number of bits to represent exactly. This is why an _expression_ like 0.1 + 0.2 == 0.3 evaluates to false! This is NOT a bug, it's a consequence that microprocessors and computer languages tend to do floating-point arithmetic in binary, not in decimal. The Number type in AS3 is the same as the 'double' type in Java and C++, which have the same issue.
 
- Gordon
 
From: [email protected] [mailto:[email protected]] On Behalf Of Boon Chew
Sent: Wednesday, March 08, 2006 9:31 AM
To: [email protected]
Subject: [flexcoders] In actionscript, converting from string to number loses precision
 
Anyone has any idea why it's not possible to convert a string to a number without losing precision?
 
var s:String = "952.86";
var i:Number = parseFloat(s);
trace(i);
trace(i - 952.86);   // Not zero!
- boon
Yahoo! Mail
Use Photomail to share photos without annoying attachments.

Yahoo! Mail
Bring photos to life! New PhotoMail makes sharing a breeze.

--
Flexcoders Mailing List
FAQ: http://groups.yahoo.com/group/flexcoders/files/flexcodersFAQ.txt
Search Archives: http://www.mail-archive.com/flexcoders%40yahoogroups.com



SPONSORED LINKS
Web site design development Computer software development Software design and development
Macromedia flex Software development best practice

YAHOO! GROUPS LINKS



Reply via email to