Ah, the perils of floating point numbers....
Even though you say you would never use rational
numbers in calculations, most accountants do.
For example, the calculators you mentioned in
your points often do fixed calculations with
the number of decimals set as a fixed option.
Similarly, many accounting programs store amounts
of money in cents. e.g. in phone call records, the
rates (cents/minute) are usually stored as 10000
times the dollar amount - and so on. Formatting
the final numbers is only a part of the problem...
- joey
At 1:03 PM +0800 3/20/07, bill lam wrote:
I think session display are also affected by rounding in dyad ": eg
9!:11[2
0.295
0.29
There are many similar numbers so that simple dyad ": alone is
inappropriate for formating numbers in reports.
r1=. <.@(+&0.5)&.(%&0.01) NB. tolerant rounding
(0j2&":"0 ; 0j2&":@r1"0) a=. 1000%~205 + 10*i.20
+----+----+
|0.20|0.21|
|0.22|0.22|
|0.23|0.23|
|0.23|0.24|
|0.25|0.25|
|0.26|0.26|
|0.27|0.27|
|0.28|0.28|
|0.28|0.29|
|0.29|0.30|
|0.30|0.31|
|0.32|0.32|
|0.33|0.33|
|0.34|0.34|
|0.34|0.35|
|0.35|0.36|
|0.36|0.37|
|0.38|0.38|
|0.39|0.39|
|0.40|0.40|
+----+----+
Roger Hui wrote:
The difference is that the dyad ": depends on the
C library routines whereas 8!:x do the programming
themselves and take more care. I'll look into it.
It is likely that a fix will make ": slower.
Thanks for the report.
----- Original Message -----
From: bill lam <[EMAIL PROTECTED]>
Date: Sunday, March 18, 2007 11:14 pm
Subject: [Jprogramming] dyad ": rounding error
On both J504 and J601, dyad ": sometimes does not round properly
on some numbers, eg.
0j2 ": 0.295
0.29
0j3 ": 0.2995
0.299
incidentally 8!:x family of J601 does not have this problem,
'0.2' (8!:0) 0.295
+----+
|0.30|
+----+
'0.2' (8!:1) 0.295
+----+
|0.30|
+----+
'0.2' (8!:2) 0.295
0.30
'0.3' (8!:2) 0.2995
0.300
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
