[ 
https://issues.apache.org/jira/browse/STATISTICS-102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Herbert resolved STATISTICS-102.
-------------------------------------
    Fix Version/s: 1.4
       Resolution: Fixed

Updated in commit: 2535c20ad85ce756c0d396151d83e7fc4e2c904c

 

> CauchyDistribution inverse probability loses precision in the tails
> -------------------------------------------------------------------
>
>                 Key: STATISTICS-102
>                 URL: https://issues.apache.org/jira/browse/STATISTICS-102
>             Project: Commons Statistics
>          Issue Type: Improvement
>          Components: distribution
>    Affects Versions: 1.3
>            Reporter: Alex Herbert
>            Priority: Major
>             Fix For: 1.4
>
>
> The Cauchy distribution computes inverse probability functions in the tails 
> computes using {{{}location + scale * Math.tan(Math.PI * (p - 0.5)){}}}.
> For a probability close to 0 or 1 the subtraction of one half cancels the 
> significant digits of {{{}p{}}}: the product {{Math.PI * (p - 0.5)}} is 
> rounded to ± π/2, so the result saturates at ± 1.633 × 10{^}16{^} whatever 
> the probability is.
> {code:java}
> final CauchyDistribution dist = CauchyDistribution.of(0.5, 2);
> dist.inverseCumulativeProbability(1e-20);           // -3.266247870639074e16, 
> expected -6.366197723675814e19
> dist.inverseCumulativeProbability(1 - 1e-16);       //  3.957875932190438e15, 
> expected  5.73416113922266e15
> dist.inverseCumulativeProbability(1e-300);          // -3.266247870639074e16, 
> expected -6.366197723675814e299
> {code}
> This is not only a saturation of the extreme tail. The accuracy decreases 
> from the middle of the tail: at {{p = 1e-5}} the value is out by 5e-12 
> relative (at {{p = 1e-10}} by 5e-7); the function is not monotonic, since 
> consecutive probabilities below {{1e-18}} can return the identical value; and 
> the median is offset by a rounding of π (1.85e-16 relative for the scale 2.1, 
> location 1.2 distribution). A caller using the quantile function for a 
> likelihood, a p-value or a rejection threshold in the tail gets a value that 
> stops responding to the input and can decrease.
> Fix: use the equivalent forms valid for the two tails, {{tan(pi (p - 1/2)) = 
> -cot(pi p)}} for {{p < 1/2}} and {{cot(pi (1 - p))}} for {{{}p > 1/2{}}}, 
> which need no cancellation.
>  



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to