### Re: [AOLSERVER] Tcl rounding question

```The only way to get what you expect is to do the rounding using integer
arithmetic.
The number 18.005 must be multiplied by 1000 to keep the implied precision.
Then do the rounding and reformat as a decimal.

TCL 8.5 has libtomath which promises arbitary precision integer arithetic
but 8.4 has wide which gives a fair bit of precision before it blows up.

Watch out
expr 46341*46341
-2147479015

I've attached some procs that I use with 8.4 to round as I would expect
without having to keep everything as an integer.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```Here is a helpful link, that explains the theory:

http://www2.hursley.ibm.com/decimal/

tom jackson

On Sunday 04 May 2008 10:20, Bernhard van Woerden wrote:
The only way to get what you expect is to do the rounding using integer
arithmetic.
The number 18.005 must be multiplied by 1000 to keep the implied precision.
Then do the rounding and reformat as a decimal.

TCL 8.5 has libtomath which promises arbitary precision integer arithetic
but 8.4 has wide which gives a fair bit of precision before it blows up.

Watch out

expr 46341*46341

-2147479015

I've attached some procs that I use with 8.4 to round as I would expect
without having to keep everything as an integer.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the
email message. You can leave the Subject: field of your email blank.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```On 2008.05.03, William Scott Jordan [EMAIL PROTECTED] wrote:
This is really more of a tcl question, but I'm hoping that someone on
the list might have an explanation.  Why does [format %.2f 18.005] round
down to 18.00 and [format %.2f 1.415] round up to 1.42?  Any
guesses?  Am I missing something obvious here?

Classic floating-point precision and rounding issue.  See:

http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding

Read the third paragraph in that section.

--
Dossy Shiobara  | [EMAIL PROTECTED] | http://dossy.org/
Panoptic Computer Network   | http://panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```On Sat, 2008-05-03 at 15:31 -0700, William Scott Jordan wrote:
Hey all!

This is really more of a tcl question, but I'm hoping that someone on
the list might have an explanation.  Why does [format %.2f 18.005] round
down to 18.00 and [format %.2f 1.415] round up to 1.42?  Any
guesses?  Am I missing something obvious here?

Hello William,

As far as I can see, it's just doing the correct math.

By convention, when you round:

1.00 = 1.0
1.01 = 1.0
1.02 = 1.0
1.03 = 1.0
1.04 = 1.0
1.05 = 1.1
1.06 = 1.1
1.07 = 1.1
1.08 = 1.1
1.09 = 1.1

and so on. This way you get .0 for 5 numbers, .1 for the other numbers,
and then the distribution is uniform. And that's not tcl-only , but they
way they teached me to round numbers in school ;-P

If you simply want to discard the decimal numbers... you'd try a method
different than format's :-)

Regards,

Juan José

﻿
-
Juan José del Río|  Comercio online / e-commerce
(+34) 616 512 340|  [EMAIL PROTECTED]

Simple Option S.L.
Tel: (+34) 951 930 122
Fax: (+34) 951 930 122
http://www.simpleoption.com

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```
William Scott Jordan wrote:

Hey all!

This is really more of a tcl question, but I'm hoping that someone on
the list might have an explanation.  Why does [format %.2f 18.005] round
down to 18.00 and [format %.2f 1.415] round up to 1.42?  Any
guesses?  Am I missing something obvious here?

Tcl version 8.4, if it matters.

Thanks!

-William

Set tcl_precision to 17 to see the fullest expansion of the value that
tcl will work with.

% set tcl_precision 17
17
% expr 18.005
18.004
% expr 1.415
1.415
%

The rounding I think is obvious at that point.

-J

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```
The plot thickens:

% format %.2f 18.0051
18.01

No ideas, though.

Bas.

On 04/05/2008, at 8:01 AM, William Scott Jordan wrote:

Hey all!

This is really more of a tcl question, but I'm hoping that someone
on the list might have an explanation.  Why does [format %.2f
18.005] round down to 18.00 and [format %.2f 1.415] round up to
1.42?  Any guesses?  Am I missing something obvious here?

Tcl version 8.4, if it matters.

Thanks!

-William

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```It does explain it, but still results are not obvious :)

For example, this code:

===
set bn [set rn 0]

for {set i 0} {\$i  1000} {incr i} {
set f \${i}.005
set r [format %.2f \$f]
set d [expr \$r - \$i]
if {\$d  0.0} {
incr bn
} else {
incr rn
}
}

puts Rounded to 0.01 in \$bn cases, to 0.0 in \$rn cases

produces results:

Rounded to 0.01 in 41 cases, to 0.00 in 959 cases

Thanks,
~ Alex.

On Sat, May 3, 2008 at 7:08 PM, Dossy Shiobara [EMAIL PROTECTED] wrote:
On 2008.05.03, William Scott Jordan [EMAIL PROTECTED] wrote:
This is really more of a tcl question, but I'm hoping that someone on
the list might have an explanation.  Why does [format %.2f 18.005] round
down to 18.00 and [format %.2f 1.415] round up to 1.42?  Any
guesses?  Am I missing something obvious here?

Classic floating-point precision and rounding issue.  See:

http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding

Read the third paragraph in that section.

--
Dossy Shiobara  | [EMAIL PROTECTED] | http://dossy.org/
Panoptic Computer Network   | http://panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL
PROTECTED] with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```Yuck.   Okay, so is there any practical work-around for getting X.XX5 to
consistently round up?  I suppose I could do something like add
0.001 to any number that I'm rounding, but that seems pretty sloppy.
Is there a best practice for dealing with this?

-William

Dossy Shiobara wrote:

On 2008.05.03, William Scott Jordan [EMAIL PROTECTED] wrote:
This is really more of a tcl question, but I'm hoping that someone on
the list might have an explanation.  Why does [format %.2f 18.005] round
down to 18.00 and [format %.2f 1.415] round up to 1.42?  Any
guesses?  Am I missing something obvious here?

Classic floating-point precision and rounding issue.  See:

http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding

Read the third paragraph in that section.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```On 2008.05.03, William Scott Jordan [EMAIL PROTECTED] wrote:
Yuck.   Okay, so is there any practical work-around for getting X.XX5 to
consistently round up?  I suppose I could do something like add
0.001 to any number that I'm rounding, but that seems pretty sloppy.
Is there a best practice for dealing with this?

Best practice?  Use integers, format as decimal value as needed.  This
is often why you'll see financial applications store money values in
cents or hundredths of a cent, i.e., a dollar is stored as either 100
(100 pennies) or 1 ... if they deal in quantities that include
fractional cents.  Only at the point where the data leaves the system,
either to a UI or to another system, do they format it with the decimal
point in place.

This approach generally eliminates all floating-point precision and
rounding issues and can even result in a performance increase in
compute-heavy applications where the machine's CPU handles integer math
faster than floating-point math.

--
Dossy Shiobara  | [EMAIL PROTECTED] | http://dossy.org/
Panoptic Computer Network   | http://panoptic.com/
He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on. (p. 70)

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```

### Re: [AOLSERVER] Tcl rounding question

```You have to remember that floating point math is done in base 2, but your are
inputing your numbers in base 10.

Another weired thing to keep in mind is that the default precision in Tcl has
changed in Tcl 8.5, so some answers are now different than they were in 8.4.

Another thing to keep in mind is that there is a difference between rounding
and actual significant digits. You should carry out your calculations with as
much precision as possible, then round to something less than that.

But, back to the question you have below. Tcl [format] uses a rounding
function, it isn't a truncation of the numbers.

If you use [expr {round(18.01)}] you get 18 (an integer).

The question is why doesn't Tcl offer a math function to round to some other
decimal precision?

The answer is simple. You should never 'round' an intermediate result. That
means you don't need this in Tcl. Final results can be formatted, and
[format] offers many more options than just the number of decimal points.

(Example of a problem with intermediate rounding:

Two step round:
round(18.449, 2) = 18.45
round(18.45, 1) = 18.5

One step round:
round(18.449, 1) = 18.4

)

tom jackson

On Saturday 03 May 2008 16:10, Bas Scheffers wrote:
The plot thickens:

% format %.2f 18.0051
18.01

No ideas, though.

Bas.

On 04/05/2008, at 8:01 AM, William Scott Jordan wrote:
Hey all!

This is really more of a tcl question, but I'm hoping that someone
on the list might have an explanation.  Why does [format %.2f
18.005] round down to 18.00 and [format %.2f 1.415] round up to
1.42?  Any guesses?  Am I missing something obvious here?

Tcl version 8.4, if it matters.

Thanks!

-William

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED]

with the

body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to
[EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the
email message. You can leave the Subject: field of your email blank.

--
AOLserver - http://www.aolserver.com/

To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.

```