It seems to me that significant digits might be a better way to go
than mathPrecision. Problem is in comparing something like
3^-6 - 3^-7 = 0 or 3^-26 - 3^-27 = 0. If that is what you
intend you would likely have a problem with the latter being true,
when for your purposes it isn't, if you are using a precision based
on number of digits, rather than number of significant digits. Or am
I misunderstanding something here?
Any comments on my function from a couple days ago?
****************************
Until the matter is finished being debated and a plan of action is
decided on, a function could be used such as
function numsAreEqual n1, n2, epsilon
return abs(n1 - n2) < epsilon
end numsAreEqual
If your testing of the immediate comparison indicates to you that
there are no roundoff problems, then use =, otherwise use this
function or your own variant with an epsilon that seems prudent.
******************************
Date: Mon, 11 May 2009 09:45:48 -0700
From: Brian Yennie <[email protected]>
Subject: Re: Math issue, isn't it?
To: How to use Revolution <[email protected]>
Message-ID: <[email protected]>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Agree on the mathPrecision value. While it is true that floating point
values are an issue in any programming language, Rev should NOT be
failing on simple comparisons with two decimal places. Computers are
limited in floating point math, but they are perfectly capable of
handling 2 decimals places!!
An alternative would be to delve into significant digits, but I'm
afraid that would cause more confusion than anything...
On May 11, 2009, at 12:11 PM, Bob Sneidar wrote:
I think we are mincing words here. It's a simple matter of changing
the way the math function works.
I understand that it's an oddity that the computer doesn't quite see
how simple the numbers are, but that happens for humans too. Take a
simple case of expressing 2/3 as a floating point decimal number,
you can't actually type out the exact value, even with unlimited
monkeys to do the typing for you. That's a case where decimal has
its limitations. Computers have different limitation cases, because
they're working in binary.
That said, having a mathPrecision equivalent to numberFormat would
solve the issue too, any tiny amounts below the mathPrecision value
would be rounded up or down.
_______________________________________________
use-revolution mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-revolution
