https://bugs.documentfoundation.org/show_bug.cgi?id=99730
Bug ID: 99730
Summary: testFuncMDETERM fails due to floating point
calculations
Product: LibreOffice
Version: 5.1.2.2 release
Hardware: x86-64 (AMD64)
OS: Linux (All)
Status: UNCONFIRMED
Severity: normal
Priority: medium
Component: Calc
Assignee: [email protected]
Reporter: [email protected]
Created attachment 124910
--> https://bugs.documentfoundation.org/attachment.cgi?id=124910&action=edit
Possible patch using a fuzzy test for zero
I get the following error when running the test suite on libreoffice-5.1.2.2
/sc/qa/unit/ucalc_formula.cxx:6841:Test::testFuncMDETERM
equality assertion failed
- Expected: 0
- Actual : 6.34413156928661e-17
- Calculation of MDETERM incorrect for singular integer matrix
ucalc_formula.cxx:6841:Assertion
Test name: Test::testFuncMDETERM
equality assertion failed
- Expected: 0
- Actual : 6.34413156928661e-17
- Calculation of MDETERM incorrect for singular integer matrix
Failures !!!
Run: 224 Failure total: 1 Failures: 1 Errors: 0
The system is x86_64 Linux with a Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz CPU
using GCC 4.8.5. The error is reproducable on my system.
Commits introducing this test are
c43442dfef3f5b21bec40054b56f406efdc5a84f
8140309d636d4a870875f2dd75ed3dfff2c0fbaf (*)
222b2f9370bfc05473249ab3f78e9adca67754bf (**)
Related bug is #32834.
The first commit introduces the test using an equality test for zero, (*) fixes
this issue by using a fuzzy test for zero, while (**) removes this test for
64bit platforms.
Note that is is a very bad idea to check for equality on floating point values
and it should always be a fuzzy test (such as in (*)). This even applies if one
can guarantee that the double was converted from an integer value (as in the
test case).
I propose to revert 222b2f9370bfc05473249ab3f78e9adca67754bf (reintroducing a
error bound of 1e-4) or applying the patch below which introduces an error
tolerance of 1e-12. (And also states the error tolerance in the error message.)
Note:
Using the LU (or LUP) decomposition to calculate the determinant of a matrix is
sometimes numerically unstable and can be wrong by an order of magnitude.
Cf section 'Limitations' of https://www.mathworks.com/help/matlab/ref/det.html.
Patch (also attached):
diff -u a/sc/qa/unit/ucalc_formula.cxx b/sc/qa/unit/ucalc_formula.cxx
--- a/sc/qa/unit/ucalc_formula.cxx
+++ b/sc/qa/unit/ucalc_formula.cxx
@@ -6847,17 +6847,10 @@
aFormulaBuffer[13] = static_cast<sal_Unicode>( '0' + nSize );
m_pDoc->SetString(aPos, aFormulaBuffer.toString());
- // On crappy 32-bit targets, presumably without extended precision on
- // interim results or optimization not catching it, this test fails
- // when comparing to 0.0, so have a narrow error margin. See also
- // commit message of 8140309d636d4a870875f2dd75ed3dfff2c0fbaf
-#if SAL_TYPES_SIZEOFPOINTER == 4
- CPPUNIT_ASSERT_DOUBLES_EQUAL_MESSAGE("Calculation of MDETERM incorrect
for singular integer matrix",
+ // Do not compare with equality instead use an error margin.
+ // This applies to both 32bit and 64bit.
+ CPPUNIT_ASSERT_DOUBLES_EQUAL_MESSAGE("Calculation of MDETERM incorrect
for singular integer matrix (error tolerance 1e-12)",
0.0, m_pDoc->GetValue(aPos), 1e-12);
-#else
- CPPUNIT_ASSERT_EQUAL_MESSAGE("Calculation of MDETERM incorrect for
singular integer matrix",
- 0.0, m_pDoc->GetValue(aPos));
-#endif
}
int aVals[] = {23, 31, 13, 12, 34, 64, 34, 31, 98, 32, 33, 63, 45, 54, 65,
76};
--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Libreoffice-bugs mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
