My buddy Mike had just written a blog posting on this - http://mikeorth.com/?p=8
--- In [email protected], "Johannes Nel" <[EMAIL PROTECTED]> wrote: > > this is well discussed and a side effect of dealing with Floats > > On Mon, Jul 21, 2008 at 12:03 PM, Kuldeep Atil <[EMAIL PROTECTED]> > wrote: > > > well, u can use number.tofixed(2); > > > > > > --- In [email protected] <flexcoders%40yahoogroups.com>, "b_alen" > > <alen.balja@> wrote: > > > > > > Here is how I arrived at this: > > > > > > var res:Number; > > > > > > var a:Number = 5; > > > var b:Number = 4.8; > > > > > > res = a - b; > > > trace("result: " + res + ", a: " + a + ", b: " + b); > > > > > > > > > The trace clearly shows that the value of a is 5, and the value of b > > > is 4.8. However the end result is clearly shown as > > 0.2000000000000018. > > > > > > Ok, I wanted to create a workaround where I will make sure that b is > > > really 4.8, and I used toPecision() method. > > > > > > var b1:String = b.toPrecision(2); > > > var b2:Number = Number(b1); > > > trace("b1: " + b1 + ", b2: " + b2); > > > > > > res = a - b2; > > > trace("result: " + res + ", b1: " + b1 + ", b2: " + b2); > > > > > > > > > Again with same result. b2 was traced as 4.8, but the end result > > still > > > showing 0.2000000000000018. As this can of course break all further > > > calculations I had to make sure result is really holding a value > > that > > > it should, based on all mathematical logic. So I did this: > > > > > > var res1:String = res.toPrecision(2); > > > res = Number(res1); > > > > > > trace("result: " + res); > > > > > > I got the expected result and the value of res is now 0.2. > > > > > > This is however very ugly, in case I'm not doing something wrong. > > > Every time we expect a floating point value in our calculation we > > have > > > to handle it to ensure the proper value is calculated, because what > > > Flash calculates is just wrong. Also, every intermediate calculation > > > has to be stored in a variable converted to String with toPrecision > > > and back to Number for future use. I really don't know how to > > properly > > > handle this and would appreciate any advice. Imagine the shopping > > cart > > > system with this sort of unpredictable behavior. > > > > > > For the end, here's one more weirdness: > > > > > > var c:Number = 488.8; > > > > > > trace("c: " + c + " , c.toPrec: " + c.toPrecision(2)); > > > > > > > > > Traces out c: 4.8, c.toPrec: 4.9e+2 > > > > > > > > > > > > > > > Thanks, > > > > > > Alen > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > var c:Number = 488.8; > > > > > > > > > > > > > -- > j:pn > \\no comment >
