On Fri, Dec 3, 2021 at 1:48 PM Alan Brown via subsurface
<[email protected]> wrote:
> Thanks for looking into this. The test still fails but the number of
> lines in the diffs are fewer.
Still seems to be that EAD line if I count the columns right.
We do have a few places where we end up converting to an integer
value, only to then convert back to a floating point.
Fore example, depth_to_bar() returns a 'double', but we do things the
wrong way around - doing the calculations in floating point in
calculate_depth_to_mbar(), but returning things as a integer 'mbar',
and then depth_to_bar() does
return depth_to_mbar(depth, dive) / 1000.0;
and I think the reason for that is historical - depth_to_bar() came
later, to do deco calculations, and it thus used the (existing)
"integer mbar" function that we use.
End result: we have some unnecessary quantization of values in the
middle of calculations.
That said, it still smells like some missing rounding, because I don't
see why you'd get different values on i387 otherwise.
And I don't see where it would be.
Linus
_______________________________________________
subsurface mailing list
[email protected]
http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface
