Re: accuracy of floating point calculations: d vs cpp
On 07/22/2019 08:48 PM, Timon Gehr wrote: > This is probably not your problem, but it may be good to know anyway: D > allows compilers to perform arbitrary "enhancement" of floating-point > precision for parts of the computation, including those performed at > compile time. I think this is stupid, but I haven't been able to > convince Walter. For completeness, at least C++ gives the same freedom to the compiler, right? And if I'm not mistaken, an additional potential problem with floating point is the state of floating point flags: they are used for all floating point computations. If a function sets them for its use and then fails to reset them to the previous state, all further computations will be affected. Ali
Re: accuracy of floating point calculations: d vs cpp
On 22.07.19 14:49, drug wrote: I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases. This is probably not your problem, but it may be good to know anyway: D allows compilers to perform arbitrary "enhancement" of floating-point precision for parts of the computation, including those performed at compile time. I think this is stupid, but I haven't been able to convince Walter.
Re: accuracy of floating point calculations: d vs cpp
22.07.2019 17:19, drug пишет: 22.07.2019 16:26, Guillaume Piolat пишет: Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different. What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.0001%. You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose. Unfortunately error has been turned out to be much bigger than I guessed before. So obviously there is a problem either on D side or on C++ side. Error is too huge to ignore it. There was a typo in C++ implementation. I did simple-n-dirt Python version and after the typo fixed all three implementations show the same result if one filter update occurs. But if several updates happen a subtle difference exists nevertheless, error accumulates somewhere else - time for numerical methods using.
Re: accuracy of floating point calculations: d vs cpp
22.07.2019 16:26, Guillaume Piolat пишет: Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different. What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.0001%. You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose. Unfortunately error has been turned out to be much bigger than I guessed before. So obviously there is a problem either on D side or on C++ side. Error is too huge to ignore it.
Re: accuracy of floating point calculations: d vs cpp
On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote: Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? This likely has little to do with the language, and more with the implementation. Basic floating point operations at the same precision should give the same results. There can be differences in float printing (see [1]) and math functions (sqrt, cos, pow etc.) however. Tips for getting consistent results between C/C++ and D: - Use the same backend, so compare DMD with DMC, LDC with CLANG and GDC with GCC. - Use the same C runtime library. On Unix glibc will likely be the default, on Windows you likely use snn.lib, libcmt.lib or msvcrt.dll. - On the D side, use core.stdc.math instead of std.math - Use the same optimizations. (Don't use -ffast-math for C) [1] https://forum.dlang.org/post/fndyoiawueefqoeob...@forum.dlang.org
Re: accuracy of floating point calculations: d vs cpp
On Monday, 22 July 2019 at 13:23:26 UTC, Guillaume Piolat wrote: On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote: I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases. Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different. What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.0001%. You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose.
Re: accuracy of floating point calculations: d vs cpp
On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote: I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases. Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.
Re: accuracy of floating point calculations: d vs cpp
On 23/07/2019 12:58 AM, drug wrote: 22.07.2019 15:53, rikki cattermole пишет: https://godbolt.org/z/EtZLG0 hmm, in short - this is my local problem? That is not how I would describe it. I would describe it as IEEE-754 doing what IEEE-754 is good at. But my point is, you can get the results to match up, if you care about it.
Re: accuracy of floating point calculations: d vs cpp
22.07.2019 15:53, rikki cattermole пишет: https://godbolt.org/z/EtZLG0 hmm, in short - this is my local problem?
Re: accuracy of floating point calculations: d vs cpp
On 23/07/2019 12:49 AM, drug wrote: I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases. https://godbolt.org/z/EtZLG0
accuracy of floating point calculations: d vs cpp
I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.
