Re: [racket-users] Re: Probabilities in log-space and rounding errors

[Laurent]

On Wed, Oct 21, 2015 at 11:24 PM, Paolo Giarrusso 
wrote:

> On 20 October 2015 at 16:44, Laurent  wrote:
> > Thanks Paolo, yes, I saw that indeed and it is most likely relevant. I
> was
> > hoping my problem would be sufficiently more specific; in particular, I'd
> > like to detect whether the value being above 0 is likely a rounding error
> > (in which case I can just round it down to 0 again) or likely a mistake
> on
> > my part in my equations or in their implementation.
>
> I'm sorry, but your dichotomy seems a fallacy. Rounding errors are the
> rule in floating point, and you need to account for them, otherwise
> you have a bug, as every introduction to numerical analysis should
> teach.
>

Thanks for your detailed explanation, Paolo. (I do use lgsum, lg1+, lg1-,
etc. wherever I can, and try to rearrange my formulas to avoid worst cases,
but I could probably use a bit more of error propagation analysis.)

Maybe I forgot a piece of information though: I'm fine with getting
sub-probabilities as the result of floating point errors, i.e. numbers that
should sum to 1 (in normal space) but where the sum is actually 0.8 for
example. That's bad but for my purposes it's not too bad. However, I'm
/not/ fine with super-probabilities. For what I do, ending with
sub-probabilities mean my results could be better if I were more careful
with my rounding errors. Super-probabilities mean I'm cheating and my
results are likely to be better than they should be.

So I was wondering if there was a way to make sure that whatever the
operations I do, those rounding errors ensure that I never end up with
super-probabilities. I was hoping that since this seems more specific than
floating point error problems in general, there may something to be done.
But maybe the former problem is just as bad as the latter.

Laurent

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: Probabilities in log-space and rounding errors

[Laurent]

Thanks Paolo, yes, I saw that indeed and it is most likely relevant. I was
hoping my problem would be sufficiently more specific; in particular, I'd
like to detect whether the value being above 0 is likely a rounding error
(in which case I can just round it down to 0 again) or likely a mistake on
my part in my equations or in their implementation.

On Tue, Oct 20, 2015 at 2:24 PM, Paolo Giarrusso 
wrote:

> On Tuesday, October 20, 2015 at 12:15:21 PM UTC+2, Laurent Orseau wrote:
> > The built-in log-space arithmetic operations are wonderful. (Thanks so
> much Neil!)
> >
> >
> > It sometimes happens that after some log-operations where the result is
> very close to 0 that it actually goes slightly above 0 as a result of
> floating point errors, and thus is not a log-probability anymore, which
> gave me a few headaches.
> >
> >
> >
> > One obvious approach is to surround the result with a check, and if it
> is positive but not, say, above 1e-8 then round it down to 0, otherwise
> raise an exception if positive.
> >
> >
> > But this feels like a hack and I was wondering if there was any "good"
> way to ensure that any correct log-space operation remains a
> log-probability  (i.e., never goes above 0).
> >
> >
> > Thanks,
> > Laurent
>
> If lg+ is involved, this discussion in the docs seems relevant — apologies
> if you've seen that already:
>
> http://docs.racket-lang.org/math/flonum.html#%28def._%28%28lib._math%2Fflonum..rkt%29._lg-%29%29
> ?
>
> After explaining that a good solution is an open research problem, docs
> state:
> >  Further, catastrophic cancellation is unavoidable when logx and logy
> themselves have error, which is by far the common case. [...] There are
> currently no reasonably fast algorithms for computing lg+ and lg- with low
> relative error. For now, if you need that kind of accuracy, use
> math/bigfloat.
>
> Disclaimer: not an expert here, might well be missing something.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: Probabilities in log-space and rounding errors

[Paolo Giarrusso]

On Tuesday, October 20, 2015 at 12:15:21 PM UTC+2, Laurent Orseau wrote:
> The built-in log-space arithmetic operations are wonderful. (Thanks so much 
> Neil!)
> 
> 
> It sometimes happens that after some log-operations where the result is very 
> close to 0 that it actually goes slightly above 0 as a result of floating 
> point errors, and thus is not a log-probability anymore, which gave me a few 
> headaches.
> 
> 
> 
> One obvious approach is to surround the result with a check, and if it is 
> positive but not, say, above 1e-8 then round it down to 0, otherwise raise an 
> exception if positive.
> 
> 
> But this feels like a hack and I was wondering if there was any "good" way to 
> ensure that any correct log-space operation remains a log-probability  (i.e., 
> never goes above 0).
> 
> 
> Thanks,
> Laurent

If lg+ is involved, this discussion in the docs seems relevant — apologies if 
you've seen that already:
http://docs.racket-lang.org/math/flonum.html#%28def._%28%28lib._math%2Fflonum..rkt%29._lg-%29%29
 ?

After explaining that a good solution is an open research problem, docs state:
>  Further, catastrophic cancellation is unavoidable when logx and logy 
> themselves have error, which is by far the common case. [...] There are 
> currently no reasonably fast algorithms for computing lg+ and lg- with low 
> relative error. For now, if you need that kind of accuracy, use math/bigfloat.

Disclaimer: not an expert here, might well be missing something.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.