Is there a better way to get the decimal value if flash is returning .2999999?
The decimal value will always be either .1, .2 or .3. It will not exceed that. I just need to test for a .3 value. Any ideas? -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of T. Michael Keesey Sent: Wednesday, December 06, 2006 3:23 PM To: Flashcoders mailing list Subject: Re: [Flashcoders] Compare Decimal Point Value? In Flash 9, I get this for the traces: 2.3 0.2999999999999998 0.3 false false true Good old-fashioned floating point precision error. Instead of testing if they are equal, You could test if the difference between them is very small, e.g.: trace(Math.abs(num2 - num3) < 0.00001); // true On 12/6/06, Doug Coning <[EMAIL PROTECTED]> wrote: > I'm scratching my head over this. Not sure what I'm doing wrong? I > need to compare a decimal value of a numeric field to see if it contains > a specific value of 0.3. Here is some sample code: > > var num:Number = 2.3; > var num2:Number = num - int(num); > var num3:Number = 0.3; > > trace(num); // 2.3 > trace(num2); // 0.3 > trace(num3); // 0.3 > trace(num2 == num3);// false > trace(num2 > num3);// false > trace(num2 < num3);// true > > Why doesn't 0.3 equal 0.3? I've replaced int with Math.floor(), and it > still doesn't work. I need to get the value after the decimal point and > see if it equals 0.3. > > Thanks for your help. > > Doug Coning > Senior Web Development Programmer > FORUM Solutions > [EMAIL PROTECTED] > > This e-mail and any attachment(s) are intended for the specified recipient(s) only and are legally protected. If you have received this communication in error, please "reply to" sender's e-mail address with notification of the error and then destroy this message in all electronic and physical forms. > _______________________________________________ > [email protected] > To change your subscription options or search the archive: > http://chattyfig.figleaf.com/mailman/listinfo/flashcoders > > Brought to you by Fig Leaf Software > Premier Authorized Adobe Consulting and Training > http://www.figleaf.com > http://training.figleaf.com > -- T. Michael Keesey The Dinosauricon: http://dino.lm.com Parry & Carney: http://parryandcarney.com ISPN Forum: http://www.phylonames.org/forum/ _______________________________________________ [email protected] To change your subscription options or search the archive: http://chattyfig.figleaf.com/mailman/listinfo/flashcoders Brought to you by Fig Leaf Software Premier Authorized Adobe Consulting and Training http://www.figleaf.com http://training.figleaf.com This e-mail and any attachment(s) are intended for the specified recipient(s) only and are legally protected. If you have received this communication in error, please "reply to" sender's e-mail address with notification of the error and then destroy this message in all electronic and physical forms. _______________________________________________ [email protected] To change your subscription options or search the archive: http://chattyfig.figleaf.com/mailman/listinfo/flashcoders Brought to you by Fig Leaf Software Premier Authorized Adobe Consulting and Training http://www.figleaf.com http://training.figleaf.com
