> On Oct 29, 2016, at 9:16 AM, David T. <[email protected]> wrote: > > John, > > Probably. As in, probably I did something wrong. (What else is new? ) > > When I first queried the database, it returned only integers when I > calculated the shares, so I Googled to find out how to get some decimals. My > solution was to multiply by 1.0, which yielded the aforementioned results. > > Specifically, my query included SUM (shares*1.0/shares_denom). This resulted > in the residuals. > > So, what is the "rational" way to calculate the shares? >
David, Rational math is what you learned in primary school: Find the least common denominator, convert every fraction to be expressed with that denominator, then add the numerators and simplify the sum. Always use integers, no decimal points. The reason that's necessary is that 1/10 isn't exactly representable in binary, so for the computer 0.1 + 0.9 - 1.0 != 0.0 at infinite precision. The error accumulates and is larger when more decimal places are involved so for more complex calculations the error can become large enough to display, as you discovered. Regards, John Ralls _______________________________________________ gnucash-devel mailing list [email protected] https://lists.gnucash.org/mailman/listinfo/gnucash-devel
