Dear OpenERP experts, I don't yet what would be wrong with the algorithm we tried to propose here. But I could see that the topic of rounding floats to the shortest decimal fraction is actually a known topic of numerical calculus. I've found some sources claiming the best algorithm would be the Dybvig's one: http://citeseer.ist.psu.edu/viewdoc/download;jsessionid=0CF999B48641E6A3CCDCCCA2D895827B?doi=10.1.1.52.2247&rep=rep1&type=pdf
Also, I've seen that Python 2.7 or 3.1 are doing such rounding to shortest decimal fraction. I don't know what they are using, but may be it's reusable somewhat. I've also found this http://research.swtch.com/2011/07/floating-point-to-decimal-conversion-is.html and this in response: http://smalltalk.gnu.org/blog/bonzinip/floating-point-decimal-conversion-not-so-easy Holly shit! -- You received this bug notification because you are a member of C2C OERPScenario, which is subscribed to OpenERP Project Group. https://bugs.launchpad.net/bugs/882036 Title: Incorrect float rounding when setting float field with too high precision Status in OpenERP GTK Client: Confirmed Status in OpenERP Server: Confirmed Status in OpenERP Server 6.0 series: Confirmed Bug description: Concerns 6.0 and trunk. If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and not 0.12. The error is in the call of the format string "%.2f"%val which introduces a mathematical error. The round function must be called to apply the correct rounding before formatting the string. It should be: "%.2f"%round(val*100)/100 Fix class digits_change of class float in the server. BUT fix also the gtk client AND web client as they all have that error (I let you find the right line) To manage notifications about this bug go to: https://bugs.launchpad.net/openobject-client/+bug/882036/+subscriptions _______________________________________________ Mailing list: https://launchpad.net/~c2c-oerpscenario Post to : [email protected] Unsubscribe : https://launchpad.net/~c2c-oerpscenario More help : https://help.launchpad.net/ListHelp
