hello Morten,
acc. to a proposal of John Denker to work around wrong floor() results for 'zombie values' by
floor(x) < x ? floor(x) : floor(x) - 1;
i'd try to put it into goffice/goffice/math/go-math.c, and it seems to work.
would you mind to check the code:
---------------------------
double
go_fake_trunc (double x)
{
x = go_d2d (x);
go_fake_trunc (double x)
{
x = go_d2d (x);
if (x == floor (x))
return x;
return x;
// edit b. wip 2021-07-09: trying to avoid wrong results for rounddown
return (x >= 0)
// ori was: ? floor (go_add_epsilon (x))
? floor ( x ) <= x ? floor( x ) : floor( x ) - 1
// ori was: : -floor (go_add_epsilon (-x));
: -floor ( -x ) >= -x ? -floor( -x ) : -floor ( -x ) - 1;
}
---------------------------
return (x >= 0)
// ori was: ? floor (go_add_epsilon (x))
? floor ( x ) <= x ? floor( x ) : floor( x ) - 1
// ori was: : -floor (go_add_epsilon (-x));
: -floor ( -x ) >= -x ? -floor( -x ) : -floor ( -x ) - 1;
}
---------------------------
if this (or a variant, i'm not! an experienced coder and open for all improving suggestions) is ok i'd like to help to implement similar for the other affected functions.
attached a sheet with the results patched / standard for a value i tested with.
best regards,
b.
---
b.
---
Gesendet: Donnerstag, 08. Juli 2021 um 00:16 Uhr
Von: "Morten Welinder" <mort...@gnome.org>
An: "Steven D'Aprano" <st...@pearwood.info>
Cc: "Gnumeric Mailing List" <gnumeric-list@gnome.org>
Betreff: Re: Re: Re: Re: deco-Math project, step 00_a: exact bin and dec 'ranges' (in gnumeric).
Von: "Morten Welinder" <mort...@gnome.org>
An: "Steven D'Aprano" <st...@pearwood.info>
Cc: "Gnumeric Mailing List" <gnumeric-list@gnome.org>
Betreff: Re: Re: Re: Re: deco-Math project, step 00_a: exact bin and dec 'ranges' (in gnumeric).
Gnumeric's rounding here is indeed done with
round_to_int(x * 10^d) / 10^d
except that round_to_int deliberately misrounds 0.5-1ulp to 1. That's
not important here.
This means that, in general, you should not expect last-bit accuracy
for this operation. Both the multiplication and the division can
introduce rounding errors. I am willing to entertain patches that fix
that if (a) someone is sufficiently interested to do the work, and (b)
can do it in a sane way that doesn't involve making strings out of
numbers. This is not a beginner's task.
But note that the function implemented would be x ->
round_to_base2(round_to_base10(x,d),53). In other words, you are
inherently double rounding. Allow me to point out that while
well-defined, it is not a very sane thing to do, except in the cases
where the outer round is known to be a no-op.
There are two types of rounding in Gnumeric: the rounding functions
such as ROUND. These change the values. Then there is the rounding
implied by attaching a format to a cell. That does not change the
value, only its display: even if you format A1 to show two decimals,
then another cell containing =A1*42 will still see the full precision
of A1.
Morten
_______________________________________________
gnumeric-list mailing list
gnumeric-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnumeric-list
round_to_int(x * 10^d) / 10^d
except that round_to_int deliberately misrounds 0.5-1ulp to 1. That's
not important here.
This means that, in general, you should not expect last-bit accuracy
for this operation. Both the multiplication and the division can
introduce rounding errors. I am willing to entertain patches that fix
that if (a) someone is sufficiently interested to do the work, and (b)
can do it in a sane way that doesn't involve making strings out of
numbers. This is not a beginner's task.
But note that the function implemented would be x ->
round_to_base2(round_to_base10(x,d),53). In other words, you are
inherently double rounding. Allow me to point out that while
well-defined, it is not a very sane thing to do, except in the cases
where the outer round is known to be a no-op.
There are two types of rounding in Gnumeric: the rounding functions
such as ROUND. These change the values. Then there is the rounding
implied by attaching a format to a cell. That does not change the
value, only its display: even if you format A1 to show two decimals,
then another cell containing =A1*42 will still see the full precision
of A1.
Morten
_______________________________________________
gnumeric-list mailing list
gnumeric-list@gnome.org
https://mail.gnome.org/mailman/listinfo/gnumeric-list
deco-math_step_00_b_sa_code_exact_rounddown_roundup_in_gnumeric.ods
Description: application/vnd.oasis.opendocument.spreadsheet
_______________________________________________ gnumeric-list mailing list gnumeric-list@gnome.org https://mail.gnome.org/mailman/listinfo/gnumeric-list
