You could also implement a decimalNumber Class that does its arithmetic
in decimal.
You need to carry guard digits through calculations to ensure that
rounding is done properly.
You can make the numbers as large as you want with whatever precision
you want.
That was one of the winning technology bits that IBM had in the 1960s
and on. Their mainframe processors (1400,360,370 series) actually
supported decimal number format with arithmetic operations that made
financial applications easier to deal with and made the COBOL decimal
structures work well. This is one of the ways that they dominated the
financial world.
Ron
JesterXL wrote:
Late in the thread, and suck at math, but there was discussion about this on
Flexcoders too. Something about how certain numbers are only so big in
Flash Player, and there was talk of porting BigInt from Java.
Might want to check the archives there, or join, and ask since more Adobe
peeps float around there.
----- Original Message -----
From: "Boon Chew" <[EMAIL PROTECTED]>
To: "Flashcoders mailing list" <[email protected]>
Sent: Wednesday, March 08, 2006 12:03 AM
Subject: Re: [Flashcoders] Doh! Converting a floating point string to
numberloses precision
I am aware of this issue in precision loss in floating point computation.
But here we are talking about converting a number from string back to
number.
Why is it so hard to implement this correctly in the VM to ensure that the
string when converted back to number is representated exactly the same as
the number version?
Out of curiosity I implemented this piece of logic in C++ and the resulting
floating point number represented in string is exactly the same after the
conversion.
Looks like there is a bug in the string to number conversion.
- boon
Ron Wheeler <[EMAIL PROTECTED]> wrote:
The way you deal with it will depend on what you want to do.
If you want to see is something is "close enough to zero" you subtract
them and check the difference
if (Math.abs(i - 952.86)<=0.1){trace("close enough for me")}else
{trace("Not the same")};
if (Math.abs(i - 952.86)<=0.0000000001){trace("Really close ")}else
{trace("Not yet the same")};
If you use enough leading zeros you will get code that never evaluates
to true so do not get wild with leading zeros.
As long as you remember that numbers that are not explicitly cast as
integers have to be treated with care, you will be alright.
This problem has been around the computer world for the last 50 years
and will be around for some time to come, so we have to just deal with it.
Ron
Yotam Laufer wrote:
-1.13686837721616e-13 is as close to zero as you get matey.... 13 places
after the decimal...
On 07/03/06, Boon Chew wrote:
How do you usually deal with the loss in precision (a bit oxymoron since
floating point can't be exactly represented) when converting a floating
point string to a number?
var s:String = "952.86";
var i:Number = parseFloat(s);
trace(i);
trace(i - 952.86); // Not zero!
- boon
---------------------------------
Yahoo! Mail
Bring photos to life! New PhotoMail makes sharing a breeze.
_______________________________________________
[email protected]
To change your subscription options or search the archive:
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Brought to you by Fig Leaf Software
Premier Authorized Adobe Consulting and Training
http://www.figleaf.com
http://training.figleaf.com
--
Yotam Laufer | Flash Developer | mobile +44 (0) 79 205 17 212 | skype:
ubermutt
------------------------------------------------------------------------
_______________________________________________
[email protected]
To change your subscription options or search the archive:
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Brought to you by Fig Leaf Software
Premier Authorized Adobe Consulting and Training
http://www.figleaf.com
http://training.figleaf.com
_______________________________________________
[email protected]
To change your subscription options or search the archive:
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Brought to you by Fig Leaf Software
Premier Authorized Adobe Consulting and Training
http://www.figleaf.com
http://training.figleaf.com
---------------------------------
Yahoo! Mail
Bring photos to life! New PhotoMail makes sharing a breeze.
---------------------------------
Yahoo! Mail
Bring photos to life! New PhotoMail makes sharing a breeze.
_______________________________________________
[email protected]
To change your subscription options or search the archive:
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Brought to you by Fig Leaf Software
Premier Authorized Adobe Consulting and Training
http://www.figleaf.com
http://training.figleaf.com
_______________________________________________
[email protected]
To change your subscription options or search the archive:
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Brought to you by Fig Leaf Software
Premier Authorized Adobe Consulting and Training
http://www.figleaf.com
http://training.figleaf.com
_______________________________________________
[email protected]
To change your subscription options or search the archive:
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Brought to you by Fig Leaf Software
Premier Authorized Adobe Consulting and Training
http://www.figleaf.com
http://training.figleaf.com
