Hi JB, The solution to the issue you have here is to always add a small 'epsilon' value to your numbers before truncating.
What is manifesting itself here is the innate approximation methods used by floating-point arithmetic, combined with the fact the trunc function does not shield you from these (indeed, trunc is there so you can use it as a basis for your own rounding methods and shouldn't really be used raw). Any floating-point number should not be considered a single number at all, but a *range* of numbers (alternatively, you can view it as a single number +/- an error). So: 34.2 is actually: 34.2 - <epsilon> to 34.2 + <epsilon> 36 is actually: 36 - <epsilon> to 36 + <epsilon> Therefore, when you get to (36 - 34.2) * 100, what you actually have is the range: 180 - epsilon to 180 + epsilon The actual representative stored in the computer can be any number within this range, and it just turns out in the case in question, the representative is: 179.99999999999971578291 All the trunc function does is chop off the fractional part, leaving 179. Therefore, in order to get the 'expected' answer, you need to ensure that you correct for maximum deviation *below* an integer that can occur before truncating. i.e. You need to add an epsilon value. As a general rule of thumb, if you require n decimal places of precision, you should use an epsilon value of 1e-(n+1). So, let's say that your calculations need no more precision than 7 decimal places, then epsilon = 0.00000001 And your code becomes: trunc((36 - 34.2) * 100 + 1e-8) Which does, indeed, return 180. I should point out that this is probably the only instance where you would get bitten by this precisely because the trunc function does nothing to correct for error in the floating-point approximation as it does not know the precision with which the number you passed to it was calculated. Warmest Regards, Mark. On Fri, 2005-10-07 at 13:16 +0200, jbv wrote: > Hi list, > > example 1 : > > put 1.8 into myA > put trunc(myA * 100) into myA > > in that case, myA=180 which is OK > ------------------ > > example 2 : > > put 34.2 into myA > put 36 into myT > > if myA > 0 then > if myA < myT then > get myA > put myT - myA into myA > put it into myT > else > put 0 into myA > end if > end if > > put trunc(myA * 100) into myA > > in that case, myA=179 ????????????????????? > > > Best, > JB > > _______________________________________________ > use-revolution mailing list > [email protected] > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-revolution ------------------------------------------------------------------ Mark Waddingham ~ [EMAIL PROTECTED] ~ http://www.runrev.com Runtime Revolution ~ User-Centric Development Tools _______________________________________________ use-revolution mailing list [email protected] Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
