I recently added a section to my book on this topic. I have inserted the text below. If this doesn't help, maybe we can figure out a better way to explain it.
Henry Rich Understanding Precision The first thing to remember is that numeric constants containing a decimal point are 64-bit floating-point numbers, not rational numbers, no matter how many decimal places you provide: 0j20 ": 1.234567890123456789 1.23456789012345670000 Floating-point numbers have at most about 16 decimal digits of precision; the additional digits were lost. If you want an exact-precision fraction, specify a rational constant or create one by dividing exact-precision integers: 1234567890123456789r1000000000000000000 1234567890123456789r1000000000000000000 1234567890123456789x % 1000000000000000000x 1234567890123456789r1000000000000000000 0j20 ": 1234567890123456789r1000000000000000000 1.23456789012345678900 NB. full precision To make a calculation use exact precision, you must make part of it exact-precision, and make sure that no floating-point values appear. This means that you must move into the exact-precision domain before you produce a fraction or an integer that will not fit exactly into a machine integer (which is 32 or 64 bits depending on your machine). 22 ": 9 ^ 20 12157665459056929000 The result did not fit into a 64-bit float, so significance was lost... 22 ": 1 + 9 ^ 20 12157665459056929000 ...with the usual floating-point effects: adding 1 does not change the value. 22 ": 1x + 9 ^ 20 12157665459056929000 Making the 1 extended doesn't help, because 9 ^ 20 has already produced a floating-point value. 22 ": 9 ^ 20x 12157665459056928801 With the 20 extended, all the computations are done in exact precision. 22 ": 1 + 9 ^ 20x 12157665459056928802 Some verbs, such as %: (square root) and ^. (natural logarithm), may produce non-integral results. The interpreter will represent the result of such a verb as an exact-precision number if possible, but if the result has no exact representation it will revert to floating-point: %: 1r4 1r2 %: 1r5 0.447214 There is no way to get an exact result from %: 2, but it is possible, with some effort, to get a result with more precision than is provided by a floating-point number. The key is the idiom <[EMAIL PROTECTED] (or >[EMAIL PROTECTED]), where v is the verb you want to apply. When you code <[EMAIL PROTECTED], the interpreter knows you are interested in only the integer part of the result, and if the operand is exact-precision, the interpreter will evaluate the integer part of the result exactly. By adjusting the size of the integer part, you can end up with high-precision fractions. 0j20 ": t =. %: 2x 1.41421356237309510000 %: 2x is a floating-point value, limited as usual to 16 decimal places or so. 0j20 ": *: t 2.00000000000000040000 Imprecise in the 17th place, as expected.. 0j20 ": t =. (10^20x) %~ <[EMAIL PROTECTED]: (10^40x) * 2x 1.41421356237309504880 Here we scaled 2 up with 40 low-order decimal zeros; then took the square root, using <[EMAIL PROTECTED] to ensure that the entire integer result is accurate; then scaled back down. 0j20 ": *: t 2.00000000000000000000 The result is more accurate. > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of p j > Sent: Saturday, March 04, 2006 10:19 AM > To: Programming forum > Subject: [Jprogramming] More precision annoyances > > below is some ways of multiplying 2 numbers. (with 20 > print precision) At the very least this type of list > should be added to the vocabulary entry for x:. This > behaviour has to be thoroughly documented and > rationalized because it is very astonishing. > > To me though, the behaviour of x: should be > "infectuous" (From python: 2*2 = 4 but 2 * 2.0 = 4.0 > - float result.). Any appearance of x: or 12x in a > sentence should propagate to all numbers touched by > the extended precision number, since I cannot conceive > of a situation where the intent of x: 1e12 f y is not > to produce a result with full precision as if it were > applied to all numbers that operate on it. (I'd > suggest 2x*1e22 = x: 2e22) I would also prefer if > x:^_1 kept the full precision of its arguments. > > 1e12 * (1e12-1) > 9.9999999999900006e23 > x:^:_1 (1000000000000 * (1e12-1)) > 9.9999999999900006e23 > x:^:_1 (x: 1e12 * x: (1e12-1)) > 9.9999999999899993e23 > ((x: 1e12) * ( 1e12-1)) > 9.9999999999899993e23 > (1000000000000x * x:(1e12-1)) > 999999999999000000000000 > (x: 1e12* x:(1e12-1)) NB. Parsing mistake :( > 4398046511099602314444247010272265625r4398046511104 > (x: 1e12)* x:(1e12-1) > 999999999999000000000000 > x:^:_1 ((x: 1e12) * x:(1e12-1)) > 9.9999999999899993e23 > > (x:1e10) * x:1e12 > 10000000000000000000000 > ((x:!.0) 1e22) > 10995116277760000526905059814453125r1099511627776 > x: 1e22 > 10995116277760000526905059814453125r1099511627776 > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com > ---------------------------------------------------------------------- > For information about J forums see > http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
