Daniel, this is a bug in Flex Builder, which occurs when displaying the values of Numbers whose value is greater than 0x7FFFFFFFFFFFFFFF. I'll get a fix in ASAP. Thanks for the feedback, and sorry for the inconvenience.
To be clear, the Flex Builder debugger's Variables view is showing the wrong value, but the code is doing the right thing, and trace() is showing the right thing. - Mike Morearty, Adobe Flex Builder team --- In flexcoders@yahoogroups.com, "Daniel Freiman" <[EMAIL PROTECTED]> wrote: > > What did I mean? I'm not sure. For some reason I thought that floats loose > precision at near the end of their range, but even if that is true, that > wouldn't explain the lose of precision that large. In any event, I retried > the following code: > > var t1:Number = Number.MAX_VALUE; > var t3:Number = Number.MAX_VALUE - 1290; > var t4:Number = Number.MAX_VALUE - 1284; > var t5:Number = Math.abs(Number.MAX_VALUE - 1290); > var t6:Number = Math.abs(Number.MAX_VALUE - 1284); > > var a1:Number = t3 - t4; > var a2:Number = Math.abs(t3 - t4); > var a3:Number = t5 - t6; > var a4:Number = (Number.MAX_VALUE - 1290) - (Number.MAX_VALUE - > 1284); > var a5:Number = Math.abs(Number.MAX_VALUE - 1290) - > Math.abs(Number.MAX_VALUE - 1284); > var a6:Number = Math.abs((Number.MAX_VALUE - 1290) - > (Number.MAX_VALUE - 1284)); > > > The debugger reported all of the first group of variable values as > "9223372036854775807 [0x7fffffffffffffff]" and the traces of these values > reported all of the values as "1.79769313486231e+308" > This is not the same results that I remember getting previously where t3 != > t4. The second group all have values of 0. When i have time I'll look into > this more. > > - Daniel Freiman > > On Wed, Jun 11, 2008 at 2:04 AM, Gordon Smith <[EMAIL PROTECTED]> wrote: > > > > Number.MAX_VALUE doesn't have enough precision to handle what I was > > trying to do. > > > > What do you mean by this? A Number, being 64 bits, actually has both more > > precision and more range than int, which is 32 bits. For example, in > > addition to storing fractional values, it can store integers much larger > > than int.MAX_VALUE. > > > > > > > > Gordon Smith > > > > Adobe Flex SDK Team > > > > > > ------------------------------ > > > > *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On > > Behalf Of *Daniel Freiman > > *Sent:* Friday, June 06, 2008 2:10 PM > > *To:* flexcoders@yahoogroups.com > > *Subject:* Re: [flexcoders] Math.abs() Limitation? > > > > > > > > I figured it out. The trace made it obvious (as opposed the debugger which > > is what I was previously using). I was using Number.MAX_VALUE, not > > int.MAX_VALUE. Number.MAX_VALUE doesn't have enough precision to handle > > what I was trying to do. > > > > Thanks, > > > > - Daniel Freiman > > > > On Fri, Jun 6, 2008 at 2:29 PM, Gordon Smith <[EMAIL PROTECTED]> wrote: > > > > What do you get when you trace out the following values? > > > > > > > > int.MAX_VALUE - 1290 > > > > Math.abs(int.MAX_VALUE - 1290) > > > > int.MAX_VALUE - 1284 > > > > Math.abs(int.MAX_VALUE - 1284) > > > > > > > > Gordon Smith > > > > Adobe Flex SDK Team > > > > > > ------------------------------ > > > > *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On > > Behalf Of *Alex Harui > > *Sent:* Friday, June 06, 2008 11:18 AM > > *To:* flexcoders@yahoogroups.com > > *Subject:* RE: [flexcoders] Math.abs() Limitation? > > > > > > > > What if you use temporary variables? > > > > > > ------------------------------ > > > > *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On > > Behalf Of *Daniel Freiman > > *Sent:* Friday, June 06, 2008 9:09 AM > > *To:* flexcoders > > *Subject:* [flexcoders] Math.abs() Limitation? > > > > > > > > According to my code the following statement returns true. > > > > Math.abs(int.MAX_VALUE - 1290) == Math.abs(int.MAX_VALUE - 1284) > > > > I tried converting everything to type Number that didn't help. Getting > > ride of the abs() makes the calculation work correctly but then I don't have > > the absolute value. > > > > Thoughts? > > > > - Daniel Freiman > > > > > > > > > > >
