On Mon, Oct 31, 2016 at 11:10 PM, test <[email protected]> wrote: > Hi, > just a simple numerical example using nicolas' implementation: > 1.199999999999999999 round: 1. > "1.2" > i'd guess this result is more or less what a beginner would expect with this > line. how would he do it with roundTo:? perhaps this way? > 1.199999999999999999 roundTo: 0.1. > "1.2000000000000002" > i'm not so sure that he would immediately see that he'd need to do it this > way: > (1.199999999999999999 roundTo: (1/10))asFloat. > "1.2" > werner >
On Tue, Nov 1, 2016 at 4:58 AM, Martin McClure <[email protected]> wrote: > tl;dr: > > 1.2000000000000002 *is* the correct answer for 1.19 roundTo: 0.1. > (1.19 roundTo: (1/10))asFloat gives a different answer because it's > rounding to a multiple of exactly 0.1, not the Float nearest to 0.1. So then why don't we *really* help the naive programmer make "x roundTo: 0.1" an error? Conceptually... Number>>roundTo: quantum quantum < 1 ifTrue: [ quantum isFraction ifFalse: [ SubtleArithmeticError signal ]. ^(self / quantum) rounded * quantum SubtleArithmeticError >> defaultAction self inform: 'Decimals have an inexact Float representation, so rounding may not give you what you expect. Us Fractions instead.'. cheers -ben
