Re: tests sc_array_functions_test fail due to floating point precision
Hi Milton, I've also opened the other test documents in sc/qa/unit/data/functions/array/fods/ using LibreOffice. The documents which fail are linest.fods and also logest.fods. The others succeed. If your build completed ignoring these tests (i.e. produced a usable office under ./instdir/), could you please open the failing documents and tell us the cell addresses on the second sheet where tests fail and the actual values they produced? Below are the numbers. Some remarks: Many errors are in the range of 1E-11 to 1E-12 but some are way larger >1E30. I assume this is due some division of a very small error. Has someone done a/some theoretical error analysis of the algorithms involved? (e.g. numeric (forward/backward) stability and error estimates similar to what is well-known for LU factorization and other algorithms.) Who/What provided the reference values -- or reformulated -- what error do these numbers have (wrt to the mathematical solution). Looking at the numbers it would be better to use ROUNDSIG instead of ROUND. ROUND(1.2345E30,4) has no effect, ROUNDSIG(1.2345E30,4) rounds to 4 significant digits, i.e. 1.234E30. ROUNDSIG was only added to Calc last February, so most unit test documents don't use it (yet). Winfried ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: tests sc_array_functions_test fail due to floating point precision
Hi Eike On December 7, 2017 at 12:40 PM Eike Rathke wrote: > On Saturday, 2017-12-02 02:47:03 -0500, Milton Vandersloot wrote: > > I've also opened the other test documents in > > sc/qa/unit/data/functions/array/fods/ using LibreOffice. The documents which > > fail are linest.fods and also logest.fods. The others succeed. > >If your build completed ignoring these tests (i.e. produced a >usable office under ./instdir/), could you please open the >failing documents and tell us the cell addresses on the second >sheet where tests fail and the actual values they produced? Below are the numbers. Some remarks: Many errors are in the range of 1E-11 to 1E-12 but some are way larger >1E30. I assume this is due some division of a very small error. Has someone done a/some theoretical error analysis of the algorithms involved? (e.g. numeric (forward/backward) stability and error estimates similar to what is well-known for LU factorization and other algorithms.) Who/What provided the reference values -- or reformulated -- what error do these numbers have (wrt to the mathematical solution). Best Milton Below are the first three columns (A-C). They are prefixed by the line number (e.g. 'l21'). At the end I added the formula of the TRUE/FALSE column. I included this as some tests round to something different than 1E-12, e.g. line 5 in linest.fods. logest.fods Sheet 2: l21: 538.89039831558538.890398315583FALSE (=ROUND(A21,12)=ROUND(B21,12)) off by 3E-12 linest.fods Sheet 2: l39: 12237.3616028624 12237.3616028623FALSE (=ROUND(A39,12)=ROUND(B39,12)) off by 1E-10 l46: 1.15420554120705E+030 5.5523702096735E+030FALSE (=ROUND(A46,12)=ROUND(B46,12)) off by 4.4E30 l66: 1.36570269395312E+030 3.73790456956461E+029 FALSE (=ROUND(A66,12)=ROUND(B66,12)) off by 1.E30 l99: 2029957.60481663 2029957.60481605FALSE (=ROUND(A99,12)=ROUND(B99,12)) off by 6.E-7 l113: 1.11356775201247E+027 7.3824164159995E+026FALSE (=ROUND(A113,12)=ROUND(B113,12)) off by 4E26 l156: 9942283.62964539 9942283.62965002FALSE (=ROUND(A156,12)=ROUND(B156,12)) off by 5E-6 l157: 11577901.4177444 11577901.4177444FALSE (=ROUND(A157,12)=ROUND(B157,12)) off by 0. This keeps me puzzling ... (increasing the number of significant digits yields zeros on both entries) l163: -108.083609022555 -108.083609022579 FALSE (=ROUND(A163,12)=ROUND(B163,12)) off by 2.4E-11 l164: 34.8435504725302 34.8435504725221FALSE (=ROUND(A164,12)=ROUND(B164,12)) off by 1.2E-11 l171: 2484916.49382756 2484916.49382742FALSE (=ROUND(A171,12)=ROUND(B171,12)) off by 1.4E-6 l184: 2.7060018272122E-14 9.25081260404044E-13FALSE (=ROUND(A184,12)=ROUND(B184,12)) off by 9E-13 l186: 4.04703891013103E+033 3.46285083263994E+030 FALSE (=ROUND(A186,12)=ROUND(B186,12)) off by 4E33 l189: 5.41123064912622E-14 1.84989825908131E-12FALSE (=ROUND(A189,12)=ROUND(B189,12)) off by 1.8E-12 l201: 786024.49976 786024.5001 FALSE (=ROUND(A201,12)=ROUND(B201,12)) off by 4E-8 l216: 786024.49976 786024.5001 FALSE (=ROUND(A216,12)=ROUND(B216,12)) off by 4E-8 ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: tests sc_array_functions_test fail due to floating point precision
Hi Milton, On Saturday, 2017-12-02 02:47:03 -0500, Milton Vandersloot wrote: > I've also opened the other test documents in > sc/qa/unit/data/functions/array/fods/ using LibreOffice. The documents which > fail are linest.fods and also logest.fods. The others succeed. If your build completed ignoring these tests (i.e. produced a usable office under ./instdir/), could you please open the failing documents and tell us the cell addresses on the second sheet where tests fail and the actual values they produced? Thanks Eike -- LibreOffice Calc developer. Number formatter stricken i18n transpositionizer. GPG key 0x6A6CD5B765632D3A - 2265 D7F3 A7B0 95CC 3918 630B 6A6C D5B7 6563 2D3A Care about Free Software, support the FSFE https://fsfe.org/support/?erack signature.asc Description: PGP signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: tests sc_array_functions_test fail due to floating point precision
On 12/02/2017 08:47 AM, Milton Vandersloot wrote: functions_test.cxx:43:Assertion Test name: ArrayFunctionsTest::testArrayFormulasFODS double equality assertion failed - Expected: 1 - Actual : 0 - Delta : 1e-14 The issue seems to be (again) floating point precision. Indeed, I've compiled LibreOffice with FMA (fused multiply-add) instructions, if I disable them, i.e. passing -ffp-contract=off to GCC, then the tests succeed. Note that FMA has slightly different floating point precision than ordinary plus and mult [0, 1]. What is ultimately missing, I think, is a spec what the acceptable results of calculations in Calc are, and tests written so that they check whether or not the calculated results are in the acceptable ranges. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: tests sc_array_functions_test fail due to floating point precision
Hi Eike On December 4, 2017 10:22 PM, Eike Rathke wrote: > From what I understood by browsing shortly gcc enables > this by default for -std=gnu* and it can be unset by > #pragma STDC FP_CONTRACT OFF > [...] > Could you try if adding the pragma to include/sal/config.h > helps? I gave it a try but GCC didn't appreciate it and threw warnings back at me: include/sal/config.h:100:0: warning: ignoring #pragma STDC FP_CONTRACT [-Wunknown-pragmas] I didn't let the compilation finish since in light of the warning I expect the outcome to be the same as without pragma. (If you think otherwise, let me know then I'll invest a couple of hours into the compilation.) The GCC version used was 6.4.0 but GCC 5 shows the same behaviour. Sadly, I currently have no GCC 7 at my disposal to test if that version supports the STDC FP_CONTRACT pragma (but I don't think it does). I took the liberty to google around a little. Here is what I've found: The GCC manual [0] (for version 6.4 but also for version 7.2) states: | Whether and how floating expressions are contracted when | not disallowed by the FP_CONTRACT pragma (C99 and C11 6.5). | | Expressions are currently only contracted if | -ffp-contract=fast, -funsafe-math-optimizations or | -ffast-math are used. This is subject to change. The C99 status page of GCC[1] reads: | standard pragmas: Not implemented. Associated command-line | options to control the state of the pragmas for the whole | translation unit are available. which hint to me that instead of the pragma we should use the command line option -ffp-contract. There is also GCC bug #37845 "gcc ignores FP_CONTRACT pragma set to OFF". The last entry is: | [...] | Note: This bug was about FP contraction being done despite | the use of: | #pragma STDC FP_CONTRACT OFF | not about the pragma being ignored (the implementation of | this pragma is not required by the ISO C standard if the | default is OFF, which is now the case with GCC). So, this | bug is really fixed. A new enhancement bug could be opened | (if not already done) for the implementation of the STDC | FP_CONTRACT pragma. So GCC does not recognize the pragma and its "default" behaviour is "FP_CONTRACT=OFF". However, GCC enables FP_CONTRACT as soon as -O2 (or higher) is enabled and the target hardware supports it. LLVM/clang seems to understand the pragma FP_CONTRACT, though [3,4] Best Milton [0] https://gcc.gnu.org/onlinedocs/gcc-6.4.0/gcc/Floating-point-implementation.html [1] https://gcc.gnu.org/c99status.html [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=37845 [3] https://reviews.llvm.org/D24481 [4] https://github.com/llvm-mirror/clang/blob/master/test/Preprocessor/pragma_unknown.c ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: tests sc_array_functions_test fail due to floating point precision
Hi Milton, On Saturday, 2017-12-02 02:47:03 -0500, Milton Vandersloot wrote: > The issue seems to be (again) floating point precision. Indeed, I've compiled > LibreOffice with FMA (fused multiply-add) instructions, if I disable them, > i.e. passing -ffp-contract=off to GCC, then the tests succeed. Note that FMA > has slightly different floating point precision than ordinary plus and mult > [0, 1]. From what I understood by browsing shortly gcc enables this by default for -std=gnu* and it can be unset by #pragma STDC FP_CONTRACT OFF https://stackoverflow.com/questions/15933100/how-to-use-fused-multiply-add-fma-instructions-with-sse-avx#15933677 Could you try if adding the pragma to include/sal/config.h helps? Thanks Eike -- LibreOffice Calc developer. Number formatter stricken i18n transpositionizer. GPG key 0x6A6CD5B765632D3A - 2265 D7F3 A7B0 95CC 3918 630B 6A6C D5B7 6563 2D3A Care about Free Software, support the FSFE https://fsfe.org/support/?erack signature.asc Description: PGP signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/libreoffice
