Conceptually, a floating point number does not represent a precise
value. Instead, it represents a value from an interval which extends
roughly half a bit lower and half a bit higher in the largest
fractional position which can't be represented in that floating point
format.

But not only is that a mouthful, the "can't be represented" part makes
it hard to work with.

One approach, though, would be to extend the range all the way into
the bit that can be represented.

To illustrate:

fmtexact=:3 :'5!:5<''y'''
ic=: 3!:4
fc=: 3!:5

bump=:4 :0
  assert. IF64
  ($y)$_2 fc 3 ic x +  _3 ic 2 fc,y
)

fmts=:3 :0
  _1 0 1 fmtexact@bump"0 _/ y
)

Using a recent example from this forum:

   fmts 11110171%100
111101.709999999992
111101.710000000006
111101.710000000021

What's interesting is that if there's a 00.. sequence which turns into
a 99.. sequence in a floating point neighbor, it's probably reasonable
to treat that sequence as an infinite stream of zeros (which can be
discarded).

(Also perhaps interesting is that the verb 'bump' takes advantage of
an intentional design feature of floating point numbers: you can
compare them as integers to determine which of two floating point
numbers is larger.)

Technically you don't even need to look for repetition - just a 0 that
turns into a 9 when you go smaller (or a 9 that turns into a 0 when
you go larger) could be sufficient to treat the digit sequence as
zeros from that point on.

I am not sure if this would deal with all cases of complex number
fuzz, but it seems like an interesting idea to play with.

-- 
Raul
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm

Reply via email to