> Number.MAX_VALUE doesn't have enough precision to handle what I was trying to do.
What do you mean by this? A Number, being 64 bits, actually has both more precision and more range than int, which is 32 bits. For example, in addition to storing fractional values, it can store integers much larger than int.MAX_VALUE. Gordon Smith Adobe Flex SDK Team ________________________________ From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Daniel Freiman Sent: Friday, June 06, 2008 2:10 PM To: [email protected] Subject: Re: [flexcoders] Math.abs() Limitation? I figured it out. The trace made it obvious (as opposed the debugger which is what I was previously using). I was using Number.MAX_VALUE, not int.MAX_VALUE. Number.MAX_VALUE doesn't have enough precision to handle what I was trying to do. Thanks, - Daniel Freiman On Fri, Jun 6, 2008 at 2:29 PM, Gordon Smith <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > wrote: What do you get when you trace out the following values? int.MAX_VALUE - 1290 Math.abs(int.MAX_VALUE - 1290) int.MAX_VALUE - 1284 Math.abs(int.MAX_VALUE - 1284) Gordon Smith Adobe Flex SDK Team ________________________________ From: [email protected] <mailto:[email protected]> [mailto:[email protected] <mailto:[email protected]> ] On Behalf Of Alex Harui Sent: Friday, June 06, 2008 11:18 AM To: [email protected] <mailto:[email protected]> Subject: RE: [flexcoders] Math.abs() Limitation? What if you use temporary variables? ________________________________ From: [email protected] <mailto:[email protected]> [mailto:[email protected] <mailto:[email protected]> ] On Behalf Of Daniel Freiman Sent: Friday, June 06, 2008 9:09 AM To: flexcoders Subject: [flexcoders] Math.abs() Limitation? According to my code the following statement returns true. Math.abs(int.MAX_VALUE - 1290) == Math.abs(int.MAX_VALUE - 1284) I tried converting everything to type Number that didn't help. Getting ride of the abs() makes the calculation work correctly but then I don't have the absolute value. Thoughts? - Daniel Freiman
