Hi, using formatted number output recently I got the following output:
1.0 printPaddedWith: $0 to: 2.1. "'01.0'"
1.0 printPaddedWith: $0 to: 2.2. "'01.00'"
1.0 printPaddedWith: $0 to: 2.3. "'01.00'" <-- wrong
1.0 printPaddedWith: $0 to: 2.4. "'01.000'" <-- wrong
1.0 printPaddedWith: $0 to: 2.5. "'01.00000'"
running Float>>printPaddedWith:to: in the debugger showed me that the reason
of this odd behaviour seems to be produced by Float>>truncated and/or
Float>>fractionPart. I can provoke errors of that kind even easier:
(2.3 fractionPart * 10) truncated. => "2" (instead of 3)
In Primitive 51 (primitiveTruncated for Float>>truncated) I read
frac = modf(rcvr, &trunc);
so the problem looks like an issue with rounding errors in Math-Library's
modf
function.
Indeed, I can reproduce the odd result of "(2.3 fractionPart * 10)
truncated."
in C++ using modf:
// g++ -lm truncate.cpp && ./a.out
#include<iostream>
#include<cmath>
using namespace std;
int main(){
double f1, f2, f3;
modf(10 * 0.3, &f1);
cout << f1 << endl;
modf(10 * modf(2.3, &f2), &f3);
cout << f3 << endl;
}
// output:
// 3 (ok)
// 2 (error - should be 3)
Question: what can be done about this?
As a matter of fact, several important methods spread over a number of
classes
(Float, Number, Morph ... being among them) are senders of Float>>truncated
or
Float>>fractionPart, so rounding errors in primitive 51 or 52 could provoke
unexpected behaviour in surprising places.
--
View this message in context:
http://forum.world.st/rounding-errors-tp3327130p3327130.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
