I'll have to double check, but I'm don't think the code was doing the right thing either, because:
Math.abs(Number.MAX_VALUE - 1290) - Math.abs(Number.MAX_VALUE - 1284); // returns 0 On Fri, Jun 13, 2008 at 1:58 PM, Mike Morearty <[EMAIL PROTECTED]> wrote: > Daniel, this is a bug in Flex Builder, which occurs when displaying the > values of Numbers > whose value is greater than 0x7FFFFFFFFFFFFFFF. I'll get a fix in ASAP. > Thanks for the > feedback, and sorry for the inconvenience. > > To be clear, the Flex Builder debugger's Variables view is showing the > wrong value, but the > code is doing the right thing, and trace() is showing the right thing. > > - Mike Morearty, Adobe Flex Builder team > > > --- In [email protected] <flexcoders%40yahoogroups.com>, "Daniel > Freiman" <[EMAIL PROTECTED]> wrote: > > > > What did I mean? I'm not sure. For some reason I thought that floats > loose > > precision at near the end of their range, but even if that is true, that > > wouldn't explain the lose of precision that large. In any event, I > retried > > the following code: > > > > var t1:Number = Number.MAX_VALUE; > > var t3:Number = Number.MAX_VALUE - 1290; > > var t4:Number = Number.MAX_VALUE - 1284; > > var t5:Number = Math.abs(Number.MAX_VALUE - 1290); > > var t6:Number = Math.abs(Number.MAX_VALUE - 1284); > > > > var a1:Number = t3 - t4; > > var a2:Number = Math.abs(t3 - t4); > > var a3:Number = t5 - t6; > > var a4:Number = (Number.MAX_VALUE - 1290) - (Number.MAX_VALUE - > > 1284); > > var a5:Number = Math.abs(Number.MAX_VALUE - 1290) - > > Math.abs(Number.MAX_VALUE - 1284); > > var a6:Number = Math.abs((Number.MAX_VALUE - 1290) - > > (Number.MAX_VALUE - 1284)); > > > > > > The debugger reported all of the first group of variable values as > > "9223372036854775807 [0x7fffffffffffffff]" and the traces of these values > > reported all of the values as "1.79769313486231e+308" > > This is not the same results that I remember getting previously where t3 > != > > t4. The second group all have values of 0. When i have time I'll look > into > > this more. > > > > - Daniel Freiman > > > > On Wed, Jun 11, 2008 at 2:04 AM, Gordon Smith <[EMAIL PROTECTED]> wrote: > > > > > > Number.MAX_VALUE doesn't have enough precision to handle what I was > > > trying to do. > > > > > > What do you mean by this? A Number, being 64 bits, actually has both > more > > > precision and more range than int, which is 32 bits. For example, in > > > addition to storing fractional values, it can store integers much > larger > > > than int.MAX_VALUE. > > > > > > > > > > > > Gordon Smith > > > > > > Adobe Flex SDK Team > > > > > > > > > ------------------------------ > > > > > > *From:* [email protected] <flexcoders%40yahoogroups.com>[mailto: > [email protected] <flexcoders%40yahoogroups.com>] *On > > > Behalf Of *Daniel Freiman > > > *Sent:* Friday, June 06, 2008 2:10 PM > > > *To:* [email protected] <flexcoders%40yahoogroups.com> > > > *Subject:* Re: [flexcoders] Math.abs() Limitation? > > > > > > > > > > > > I figured it out. The trace made it obvious (as opposed the debugger > which > > > is what I was previously using). I was using Number.MAX_VALUE, not > > > int.MAX_VALUE. Number.MAX_VALUE doesn't have enough precision to handle > > > what I was trying to do. > > > > > > Thanks, > > > > > > - Daniel Freiman > > > > > > On Fri, Jun 6, 2008 at 2:29 PM, Gordon Smith <[EMAIL PROTECTED]> wrote: > > > > > > What do you get when you trace out the following values? > > > > > > > > > > > > int.MAX_VALUE - 1290 > > > > > > Math.abs(int.MAX_VALUE - 1290) > > > > > > int.MAX_VALUE - 1284 > > > > > > Math.abs(int.MAX_VALUE - 1284) > > > > > > > > > > > > Gordon Smith > > > > > > Adobe Flex SDK Team > > > > > > > > > ------------------------------ > > > > > > *From:* [email protected] <flexcoders%40yahoogroups.com>[mailto: > [email protected] <flexcoders%40yahoogroups.com>] *On > > > Behalf Of *Alex Harui > > > *Sent:* Friday, June 06, 2008 11:18 AM > > > *To:* [email protected] <flexcoders%40yahoogroups.com> > > > *Subject:* RE: [flexcoders] Math.abs() Limitation? > > > > > > > > > > > > What if you use temporary variables? > > > > > > > > > ------------------------------ > > > > > > *From:* [email protected] <flexcoders%40yahoogroups.com>[mailto: > [email protected] <flexcoders%40yahoogroups.com>] *On > > > Behalf Of *Daniel Freiman > > > *Sent:* Friday, June 06, 2008 9:09 AM > > > *To:* flexcoders > > > *Subject:* [flexcoders] Math.abs() Limitation? > > > > > > > > > > > > According to my code the following statement returns true. > > > > > > Math.abs(int.MAX_VALUE - 1290) == Math.abs(int.MAX_VALUE - 1284) > > > > > > I tried converting everything to type Number that didn't help. Getting > > > ride of the abs() makes the calculation work correctly but then I don't > have > > > the absolute value. > > > > > > Thoughts? > > > > > > - Daniel Freiman > > > > > > > > > > > > > > > > > > > >
