Oh, you mean like this -
0j2 ": 295r1000 2995r10000
0.30 0.30
0j20 ": 295r1000 2995r10000
0.29500000000000000000 0.29950000000000000000
I agree with your expectations - but seems that already works...
Pocket calculators are authoritative about math rulse? This is
any old 4 function chip, or some specific one?
- joey
At 12:35 AM +0800 3/20/07, bill lam wrote:
I agree that technically 0.295 could be rounded to 0.29, but I think
that most people (programmer or non-programmer) will expect that
rounding the quotient (295/1000) to 2 decimal places is 0.30
I also double checked with a pocket calculator.
Joey K Tuttle wrote:
It would be a shame to slow down ": ... consider
0j20 ": 0.295 0.2995
0.29499999999999998446 0.29949999999999998845
I suppose that 8!:x implements the "standard rule" that
rounding a number ending (but not really in this case) 5
depends on whether the prior digit is odd or even. Much
of this seems arbitrary at best.
- joey
At 8:45 AM -0700 3/19/07, Roger Hui wrote:
The difference is that the dyad ": depends on the
C library routines whereas 8!:x do the programming
themselves and take more care. I'll look into it.
It is likely that a fix will make ": slower.
Thanks for the report.
----- Original Message -----
From: bill lam <[EMAIL PROTECTED]>
Date: Sunday, March 18, 2007 11:14 pm
Subject: [Jprogramming] dyad ": rounding error
On both J504 and J601, dyad ": sometimes does not round properly
on some
numbers, eg.
0j2 ": 0.295
0.29
0j3 ": 0.2995
0.299
incidentally 8!:x family of J601 does not have this problem,
'0.2' (8!:0) 0.295
+----+
|0.30|
+----+
'0.2' (8!:1) 0.295
+----+
|0.30|
+----+
'0.2' (8!:2) 0.295
0.30
'0.3' (8!:2) 0.2995
0.300
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
