I think session display are also affected by rounding in dyad ": eg
9!:11[2
0.295
0.29
There are many similar numbers so that simple dyad ": alone is inappropriate for
formating numbers in reports.
r1=. <.@(+&0.5)&.(%&0.01) NB. tolerant rounding
(0j2&":"0 ; 0j2&":@r1"0) a=. 1000%~205 + 10*i.20
+----+----+
|0.20|0.21|
|0.22|0.22|
|0.23|0.23|
|0.23|0.24|
|0.25|0.25|
|0.26|0.26|
|0.27|0.27|
|0.28|0.28|
|0.28|0.29|
|0.29|0.30|
|0.30|0.31|
|0.32|0.32|
|0.33|0.33|
|0.34|0.34|
|0.34|0.35|
|0.35|0.36|
|0.36|0.37|
|0.38|0.38|
|0.39|0.39|
|0.40|0.40|
+----+----+
Roger Hui wrote:
The difference is that the dyad ": depends on the
C library routines whereas 8!:x do the programming
themselves and take more care. I'll look into it.
It is likely that a fix will make ": slower.
Thanks for the report.
----- Original Message -----
From: bill lam <[EMAIL PROTECTED]>
Date: Sunday, March 18, 2007 11:14 pm
Subject: [Jprogramming] dyad ": rounding error
On both J504 and J601, dyad ": sometimes does not round properly
on some
numbers, eg.
0j2 ": 0.295
0.29
0j3 ": 0.2995
0.299
incidentally 8!:x family of J601 does not have this problem,
'0.2' (8!:0) 0.295
+----+
|0.30|
+----+
'0.2' (8!:1) 0.295
+----+
|0.30|
+----+
'0.2' (8!:2) 0.295
0.30
'0.3' (8!:2) 0.2995
0.300
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
--
regards,
bill
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
