Yes, toFixed() is a better option. It still does not solve the problem as we don't know to what value to fix the numbers to achieve consistent behavior. In practice using something like toFixed(8) for every Number should do the job, but then again we can't be sure.
I also found this blog that seems useful: http://www.zeuslabs.us/2007/01/30/flash-floating-point-number-errors/ On Mon, Jul 21, 2008 at 3:33 PM, Kuldeep Atil <[EMAIL PROTECTED]> wrote: > well, u can use number.tofixed(2); > > > --- In flexcoders@yahoogroups.com <flexcoders%40yahoogroups.com>, "b_alen" > <[EMAIL PROTECTED]> wrote: > > > > Here is how I arrived at this: > > > > var res:Number; > > > > var a:Number = 5; > > var b:Number = 4.8; > > > > res = a - b; > > trace("result: " + res + ", a: " + a + ", b: " + b); > > > > > > The trace clearly shows that the value of a is 5, and the value of b > > is 4.8. However the end result is clearly shown as > 0.2000000000000018. > > > > Ok, I wanted to create a workaround where I will make sure that b is > > really 4.8, and I used toPecision() method. > > > > var b1:String = b.toPrecision(2); > > var b2:Number = Number(b1); > > trace("b1: " + b1 + ", b2: " + b2); > > > > res = a - b2; > > trace("result: " + res + ", b1: " + b1 + ", b2: " + b2); > > > > > > Again with same result. b2 was traced as 4.8, but the end result > still > > showing 0.2000000000000018. As this can of course break all further > > calculations I had to make sure result is really holding a value > that > > it should, based on all mathematical logic. So I did this: > > > > var res1:String = res.toPrecision(2); > > res = Number(res1); > > > > trace("result: " + res); > > > > I got the expected result and the value of res is now 0.2. > > > > This is however very ugly, in case I'm not doing something wrong. > > Every time we expect a floating point value in our calculation we > have > > to handle it to ensure the proper value is calculated, because what > > Flash calculates is just wrong. Also, every intermediate calculation > > has to be stored in a variable converted to String with toPrecision > > and back to Number for future use. I really don't know how to > properly > > handle this and would appreciate any advice. Imagine the shopping > cart > > system with this sort of unpredictable behavior. > > > > For the end, here's one more weirdness: > > > > var c:Number = 488.8; > > > > trace("c: " + c + " , c.toPrec: " + c.toPrecision(2)); > > > > > > Traces out c: 4.8, c.toPrec: 4.9e+2 > > > > > > > > > > Thanks, > > > > Alen > > > > > > > > > > > > > > > > > > > > > > var c:Number = 488.8; > > > > >
