Oleg Kobchenko wrote: > >> Does this help? > Yes, thank you. > > In fact I had come up with a somewhat similar approach. > Very straightforward ideas: separate whole and fractional > parts, multiply fractional by 16^desired precision and round, > convert each part and combine with '.' accounting for sign. > > --- Raul Miller <[EMAIL PROTECTED]> wrote: > > >> This leaves us with a trailing zero (or a trailing digit or trailing >> garbage for >> some edge cases), but that can be dealt with at a higher level. I also >> arbitrarily limited the precision of this mechanism to 9 trailing digits, >> and >> in a more general context this should instead be an estimation of the >> digits needed for the representation based on the structure of floating >> point numbers. > > I agree with both posts. One thing should be however emphasized: the leading zeros of the fractional part are significant. So then there is a small (1 in 16 numbers) error in hff:
2 hff 10.5 a.80 NB. Exact. 2 hff 10.030625 a.8 NB. Instead of the exact a.08 NB. hff then should still correctly work for 11259375.071111083 6 hff 11259375.071111083 abcdef.123456 NB. Two more numbers to consider in this context are: 7 hff 11259375.07111111 abcdef.1234567 8 hff 11259375.071111111 abcdef.12345678 -- View this message in context: http://www.nabble.com/Format-fractional-hexadecimal-tf4895825s24193.html#a14131599 Sent from the J Programming mailing list archive at Nabble.com. ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
