Robert Helling via subsurface <[email protected]> writes:
> this is an excellent observation. I completely missed the fact that for no > good reason, the fraction fn2 and fhe are rounded to integers. As > intermediate values, they should be kept double. While I > was at it, I made MOD, END, EAD, EAAD integers since those are all depths and > thus they should be mm in ints. > > My hope would now be that rounded to mm, the difference in floating point > precision should not really show up (at least in most of the cases). > > Alan, would you please check if my PR passes the tests on your architectures? > > https://github.com/subsurface/subsurface/pull/3357 > Robert, Thanks for looking into this. The test still fails but the number of lines in the diffs are fewer. https://gist.github.com/adbrown101/db61dd3c61861846f3c5f8a573d974ef https://gist.github.com/adbrown101/b5ce9440827e3d3ff975beb39f93e32d It looks like the numbers are out by 10. Regards Alan _______________________________________________ subsurface mailing list [email protected] http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface
