https://bugs.documentfoundation.org/show_bug.cgi?id=55960
--- Comment #16 from Mike Kaganski <mikekagan...@hotmail.com> ---
In double precision,
> 0.05*39=1.9500000000000002
> -2+0.05*39=-0.049999999999999822
First idea is that if we returned not only the increment from the approxDiff,
but also the number of decimals for rounding (which is 15 in this case), and
then used the rounding in the following calculation of the resulting increment
for each cell, then this specific sequence would be OK.
Yet, this is not a full solution, since this would fail with longer sequences,
like "-22;-21.95;...". That is because when multiplying the atomic increment,
we also need to decrease resulting absolute precision proportionally (so if
initially it's 5E-16 for 0.05, then for 0.05*39, it's 5*E-16*39=1.95E-14, so we
need to round the result of 0.05*39 to 14 decimals).
Eike: if agreed, I suggest to make this an easy hack.
--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Libreoffice-bugs mailing list
Libreoffice-bugs@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
