Hi Folks -- I have written some code to deal with the following issue:
It turns out that for some floats, converting to a complex number and then back to a float loses precision. The CNS (complex number string) representation doesn't use enough digits. If this is not obvious, a demonstration is here: http://www.av8n.com/computer/complex-precision.gnumeric As is my custom, I include two sheets: One with live calculations, and one with the results pasted by value, to make sure everybody can see what I'm seeing, even if there are version-to-version variations. Using one more digit makes the problem go away. Here's the basic conceptual issue: +/* Note there are two different questions: + A double can safely represent how many digits? GNM_DIG + How many digits can safely represent a double? GNM_DIG_SAFE + */ +#define GNM_DIG_SAFE (1+GNM_DIG) Right now the code to deal with this is bundled in with the imfourier patch, but I suppose it could be split out if anybody wanted. Look for references to GNM_DIG_SAFE here: http://www.av8n.com/computer/imfourier.patch This is a patch against the current 1.12.1 tarball. This counts as "mostly" fixed because I still need to track down everybody who calls the old code and see whether they need to be calling the new code. ============= See also next message. _______________________________________________ gnumeric-list mailing list gnumeric-list@gnome.org https://mail.gnome.org/mailman/listinfo/gnumeric-list
