You have to remember that floating point math is done in base 2, but your are
inputing your numbers in base 10.
Another weired thing to keep in mind is that the default precision in Tcl has
changed in Tcl 8.5, so some answers are now different than they were in 8.4.
Another thing to keep in mind is that there is a difference between rounding
and actual significant digits. You should carry out your calculations with as
much precision as possible, then round to something less than that.
But, back to the question you have below. Tcl [format] uses a rounding
function, it isn't a truncation of the numbers.
If you use [expr {round(18.01)}] you get 18 (an integer).
The question is why doesn't Tcl offer a math function to round to some other
decimal precision?
The answer is simple. You should never 'round' an intermediate result. That
means you don't need this in Tcl. Final results can be formatted, and
[format] offers many more options than just the number of decimal points.
(Example of a problem with intermediate rounding:
Two step round:
round(18.449, 2) = 18.45
round(18.45, 1) = 18.5
One step round:
round(18.449, 1) = 18.4
)
tom jackson
On Saturday 03 May 2008 16:10, Bas Scheffers wrote:
> The plot thickens:
>
> % format %.2f 18.0051
> 18.01
>
> No ideas, though.
>
> Bas.
>
> On 04/05/2008, at 8:01 AM, William Scott Jordan wrote:
> > Hey all!
> >
> > This is really more of a tcl question, but I'm hoping that someone
> > on the list might have an explanation. Why does [format %.2f
> > 18.005] round down to "18.00" and [format %.2f 1.415] round up to
> > "1.42"? Any guesses? Am I missing something obvious here?
> >
> > Tcl version 8.4, if it matters.
> >
> > Thanks!
> >
> > -William
> >
> >
> > --
> > AOLserver - http://www.aolserver.com/
> >
> > To Remove yourself from this list, simply send an email to
> > <[EMAIL PROTECTED]
> >
> > > with the
> >
> > body of "SIGNOFF AOLSERVER" in the email message. You can leave the
> > Subject: field of your email blank.
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to
> <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the
> email message. You can leave the Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
