https://bugs.documentfoundation.org/show_bug.cgi?id=55960

--- Comment #34 from Melove Gupta <[email protected]> ---
Submitted a patch: https://gerrit.libreoffice.org/c/core/+/201071
While investigating tdf#55960, I found that the floating-point artifacts during
autofill appear to originate from the linear series calculation in
"ScTable::FillSeries".

I implemented a potential fix by canonicalizing the computed value using
"rtl::math::approxValue" before storing it, which removes artifacts like
"2.22e-16" appearing near zero when filling sequences such as "-2, -1.95, ...".

I've submitted a patch but I wanted to confirm whether this is the preferred
approach for handling floating-point noise in FillSeries, or if there is a
better location or mechanism in the Calc codebase where such canonicalization
should occur.

I also added a regression unit test to ensure the issue does not reappear.

-- 
You are receiving this mail because:
You are the assignee for the bug.

Reply via email to