I was thinking of a new property called MathPrecision or something.
Set the mathprecision to -2 would round to 2 decimal places for the
result of any math equasion. Set the mathprecision to 2 would round to
the nearest 100. The default could be -9 in which case the prior error
in floating point math that started this thread would not have occurred.
Bob Sneidar
IT Manager
Logos Management
Calvary Chapel CM
On May 11, 2009, at 9:45 AM, Brian Yennie wrote:
Agree on the mathPrecision value. While it is true that floating point
values are an issue in any programming language, Rev should NOT be
failing on simple comparisons with two decimal places. Computers are
limited in floating point math, but they are perfectly capable of
handling 2 decimals places!!
An alternative would be to delve into significant digits, but I'm
afraid that would cause more confusion than anything...
On May 11, 2009, at 12:11 PM, Bob Sneidar wrote:
I think we are mincing words here. It's a simple matter of changing
the way the math function works.
I understand that it's an oddity that the computer doesn't quite see
how simple the numbers are, but that happens for humans too. Take a
simple case of expressing 2/3 as a floating point decimal number,
you can't actually type out the exact value, even with unlimited
monkeys to do the typing for you. That's a case where decimal has
its limitations. Computers have different limitation cases, because
they're working in binary.
That said, having a mathPrecision equivalent to numberFormat would
solve the issue too, any tiny amounts below the mathPrecision value
would be rounded up or down.
_______________________________________________
use-revolution mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-revolution
_______________________________________________
use-revolution mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-revolution
