On Mon, Feb 2, 2009 at 2:32 PM, Gregory Maxwell <[email protected]> wrote: > On Sat, Jan 31, 2009 at 8:33 PM, Robert Rohde <[email protected]> wrote: >> This discussion is getting side tracked. >> >> The real complaint here is that >> >> {{#expr:(0.00007 * 1000 * 1000) mod 1000}} is giving 69 when it should give >> 70. >> >> This is NOT a formatting issue, but rather it is bug in the #expr >> parser function, presumably caused by some kind of round-off error. > > It's a bug in the user's understanding of floating point on computers, > combined with % being (quite naturally) an operator on integers. > > 0.000070000… does not exist in your finite precision base-2 based computer. > > I don't think it's reasonable for Mediawiki to include a full radix-n > multi-precision floating point library in order to capture the > expected your behavior for these cases, any more than it would be > reasonable to expect it to contain a full computer algebra system so > it could handle manipulations of irrationals precisely.
I've already written code that converts 4.9999999998 to 5 immediately before performing operations that explicitly expect integers by applying an explicit 1 part in 10^10 tolerance. That covers a wide range of cases that might be affected by round-off errors while adding little overhead. -Robert Rohde _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
