Bill Murphy created MATH-1257:
---------------------------------
Summary: NormalDistribution.cumulativeProbability() suffers from
cancellation
Key: MATH-1257
URL: https://issues.apache.org/jira/browse/MATH-1257
Project: Commons Math
Issue Type: Bug
Affects Versions: 3.5
Reporter: Bill Murphy
Priority: Minor
I see the following around line 194:
{noformat}
return 0.5 * (1 + Erf.erf(dev / (standardDeviation * SQRT2)));
{noformat}
When erf() returns a very small value, this cancels in the addition with the
"1.0" which leads to poor precision in the results.
I would suggest changing this line to read more like:
{noformat}
return 0.5 * Erf.erfc( -dev / SQRT2 );
{noformat}
Should you want some test cases for "extreme values" (one might argue that
within 10 standard deviations isn't all that extreme) then you can check the
following: http://www.jstatsoft.org/v52/i07/ then look in the v52i07-xls.zip at
replication-01-distribution-standard-normal.xls
I think you will also find that evaluation of expressions such as
{noformat}NormalDistribution( 0, 1 ).cumulativeProbability( 10.0 );{noformat}
are pretty far off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
