nealef opened a new issue #4241: URL: https://github.com/apache/incubator-nuttx/issues/4241
### Summary ### I have encountered a couple of floating point operation problems. * `fmod()` returns incorrect value * `log(x)` never returns for specific values of x (see comment in https://github.com/apache/incubator-nuttx/pull/4115) ### Configuration ### **Hardware:** stm32f777zit6 **NuttX:** Based on commit 2e80d5e38ed92f67767d87b544894cdc254f9459 ### Test case ### ``` #include <stdio.h> #include <math.h> int main(int argc, char **argv) { register double z = fmod(8.0, 3.0); printf("mod: %g (0x%016llx)\n",z,z); double x = log(318.0); printf("log: %g\n",x); } ``` #### Expected Results #### ``` mod: 2 (0x4000000000000000) log: 5.76205 ``` #### Actual Results #### ``` mod: 2 (0x3ffffffffffffffe) <never returns> ``` #### Analysis #### The `fmod()` function returns an incorrect value (1.9999999999999996) that is corrected with the `printf()` processing but if that value is used in other calculations or comparisons then the wrong result may be produced. Using `gdb` to trace the result returned confirms the 1.999... value is being returned. For the second case,`gdb` shows the `log()` function is never returning as the condition: ``` while (y > y_old + epsilon || y < y_old - epsilon) ``` Is always true. The value `y` keeps swapping between 5.7620513827801778 and 5.7620513827801769. ### Circumvention ### I added some oscillation detection code which checks if the y equals the y before last: ``` double log(double x) { double y; double y_old; double y_very_old; <--- double ey; double epsilon; int relax_factor; int iter; y = 0.0; y_old = 1.0; y_very_old = 2.0; epsilon = DBL_EPSILON; iter = 0; relax_factor = 1; while (y > y_old + epsilon || y < y_old - epsilon && y != y_very_old) <--- { y_very_old = y_old; <--- y_old = y; ``` With this, perhaps naive, approach we exit with the value expected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
