[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[
https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000664#comment-16000664
]
Gilles commented on MATH-1381:
--
Please do! Thanks to both for your work on this.
Refactoring/modularizing the {{o.a.c.m.stat}} package would be a worthy goal
IMHO.
A lot of work... ;)
Unless I'm mistaken, the bulk of the original CM code was comprised of the
statistical utilities. It should have been named "Commons Stat" (to avoid
open-ended extensions).
> BinomialTest P-value > 1
>
>
> Key: MATH-1381
> URL: https://issues.apache.org/jira/browse/MATH-1381
> Project: Commons Math
> Issue Type: Bug
>Reporter: Wang Qiang
> Fix For: 4.0
>
>
> When I use the Binomial Test, I got p-value > 1 for two sided check.
> Example:
> (new BinomialTest()).binomialTest(200, 200, 0.9950429,
> AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435
> In my case, if the expected p-value is 1 (calculated by package in other
> language, scipy in this case), the p-value returned could be > 1
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[ https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000422#comment-16000422 ] Bruno P. Kinoshita commented on MATH-1381: -- Pull request looks good. Planning to merge the PR tomorrow. WDYT [~erans]? > BinomialTest P-value > 1 > > > Key: MATH-1381 > URL: https://issues.apache.org/jira/browse/MATH-1381 > Project: Commons Math > Issue Type: Bug >Reporter: Wang Qiang > Fix For: 4.0 > > > When I use the Binomial Test, I got p-value > 1 for two sided check. > Example: > (new BinomialTest()).binomialTest(200, 200, 0.9950429, > AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435 > In my case, if the expected p-value is 1 (calculated by package in other > language, scipy in this case), the p-value returned could be > 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[ https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1538#comment-1538 ] Kexin Xie commented on MATH-1381: - Thanks, [~kinow]! I've updated https://github.com/apache/commons-math/pull/43 to contain your changes. > BinomialTest P-value > 1 > > > Key: MATH-1381 > URL: https://issues.apache.org/jira/browse/MATH-1381 > Project: Commons Math > Issue Type: Bug >Reporter: Wang Qiang > Fix For: 4.0 > > > When I use the Binomial Test, I got p-value > 1 for two sided check. > Example: > (new BinomialTest()).binomialTest(200, 200, 0.9950429, > AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435 > In my case, if the expected p-value is 1 (calculated by package in other > language, scipy in this case), the p-value returned could be > 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[ https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999838#comment-15999838 ] Bruno P. Kinoshita commented on MATH-1381: -- Ooohh, makes sense, should have thought about looking at the issues. Nice catch [~erans]. [~kexinxie], let's merge our pull requests? I think mine has the solution applied to that fork project. Yours has more test cases. Let me know if you prefer to update your pull request or mine in the next days. Feel free to copy the code changed and simply apply to yours if you'd like ;-) Then we merge it and fix this issue. Cheers Bruno > BinomialTest P-value > 1 > > > Key: MATH-1381 > URL: https://issues.apache.org/jira/browse/MATH-1381 > Project: Commons Math > Issue Type: Bug >Reporter: Wang Qiang > Fix For: 4.0 > > > When I use the Binomial Test, I got p-value > 1 for two sided check. > Example: > (new BinomialTest()).binomialTest(200, 200, 0.9950429, > AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435 > In my case, if the expected p-value is 1 (calculated by package in other > language, scipy in this case), the p-value returned could be > 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[ https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999795#comment-15999795 ] Gilles commented on MATH-1381: -- Hi Bruno. Thanks for your help. But it's so sad that you had to spend time looking for clues about an [issue that was solved 9 months ago in the CM fork|https://github.com/Hipparchus-Math/hipparchus/issues/9]. :( > BinomialTest P-value > 1 > > > Key: MATH-1381 > URL: https://issues.apache.org/jira/browse/MATH-1381 > Project: Commons Math > Issue Type: Bug >Reporter: Wang Qiang > Fix For: 4.0 > > > When I use the Binomial Test, I got p-value > 1 for two sided check. > Example: > (new BinomialTest()).binomialTest(200, 200, 0.9950429, > AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435 > In my case, if the expected p-value is 1 (calculated by package in other > language, scipy in this case), the p-value returned could be > 1 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[
https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999737#comment-15999737
]
Bruno P. Kinoshita commented on MATH-1381:
--
Agree with [~erans] on spending some more time reviewing the current
implementation.
Today I spent some time reviewing distributions, binomial, and tests. Then
looked at our BinomialTest, but still couldn't tell what was wrong. Then looked
at other Java implementations, until stumbled across
[hipparchus|https://github.com/Hipparchus-Math/hipparchus]. Their BinomialTest
implementation has an extra if, to check when both extreme values are equals.
https://github.com/Hipparchus-Math/hipparchus/blob/ff26617bd14472fa84d70e8efe66c1177f952145/hipparchus-stat/src/main/java/org/hipparchus/stat/inference/BinomialTest.java#L131
The comment next to it says
{quote}One side can't move{quote}
And from our Javadocs:
{quote}The lower value is added to the p-Value (if both values are equal, both
are added). Then we continue with the next extreme value, until we added the
value for the actual observed sample.{quote}
Nothing obvious for me, but with that check, we get precisely 1.0 as p-value
for the example reported in this issue.
Added the pull request here: https://github.com/apache/commons-math/pull/59
But probably worth combining with the tests from
https://github.com/apache/commons-math/pull/43.
Hope that helps
Bruno
> BinomialTest P-value > 1
>
>
> Key: MATH-1381
> URL: https://issues.apache.org/jira/browse/MATH-1381
> Project: Commons Math
> Issue Type: Bug
>Reporter: Wang Qiang
> Fix For: 4.0
>
>
> When I use the Binomial Test, I got p-value > 1 for two sided check.
> Example:
> (new BinomialTest()).binomialTest(200, 200, 0.9950429,
> AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435
> In my case, if the expected p-value is 1 (calculated by package in other
> language, scipy in this case), the p-value returned could be > 1
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[
https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453154#comment-15453154
]
Kexin Xie commented on MATH-1381:
-
Hi [~erans], thanks for looking at the PR. I agree with you that this does
seems like it's a dirty fix and mask a potential bug in the computation.
However, the main problem here is that there is one corner case that the
current algorithm did not consider. Which is that if the probability is large
enough and the success is the same as the number of trials and both numbers are
small enough, it will cause the {{criticalValueLow}} to rise too quickly and be
the same as {{criticalValueHigh}}. The if condition in L138 is suppose to check
the symmetry case when {{pLow == pHigh}}, but is not for the case when
{{criticalValueLow == criticalValueHigh}}. At that point the probability will
always jump to above 1.
It may seem like a dirty fix, but I have checked against results in R, and
Python's scipy equivalent, and they produce the same value. I implemented this
way because it actually works in handling this boundary condition, and it's the
least change to the original implementation. Note that Python's scipy also uses
a similar approach to deal with estimated value rising above 1
https://github.com/scipy/scipy/blob/v0.14.0/scipy/stats/morestats.py#L1661
I've also updated with more exhaustive test cases, please have a look again.
> BinomialTest P-value > 1
>
>
> Key: MATH-1381
> URL: https://issues.apache.org/jira/browse/MATH-1381
> Project: Commons Math
> Issue Type: Bug
>Reporter: Wang Qiang
>
> When I use the Binomial Test, I got p-value > 1 for two sided check.
> Example:
> (new BinomialTest()).binomialTest(200, 200, 0.9950429,
> AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435
> In my case, if the expected p-value is 1 (calculated by package in other
> language, scipy in this case), the p-value returned could be > 1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[ https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451537#comment-15451537 ] Gilles commented on MATH-1381: -- Having just looked at your proposed, I can't be sure that it fixes a potential bug in the computation. The wrong value being so much larger than the expected 1, it seems quite possible that wrong results could also be lower than 1, which your fix wouldn't fix. At first sight, it would be useful to test a whole range of values against the results provided by another implementation. > BinomialTest P-value > 1 > > > Key: MATH-1381 > URL: https://issues.apache.org/jira/browse/MATH-1381 > Project: Commons Math > Issue Type: Bug >Reporter: Wang Qiang > > When I use the Binomial Test, I got p-value > 1 for two sided check. > Example: > (new BinomialTest()).binomialTest(200, 200, 0.9950429, > AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435 > In my case, if the expected p-value is 1 (calculated by package in other > language, scipy in this case), the p-value returned could be > 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MATH-1381) BinomialTest P-value > 1
[ https://issues.apache.org/jira/browse/MATH-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450960#comment-15450960 ] Kexin Xie commented on MATH-1381: - I have a fix in https://github.com/apache/commons-math/pull/43 > BinomialTest P-value > 1 > > > Key: MATH-1381 > URL: https://issues.apache.org/jira/browse/MATH-1381 > Project: Commons Math > Issue Type: Bug >Reporter: Wang Qiang > > When I use the Binomial Test, I got p-value > 1 for two sided check. > Example: > (new BinomialTest()).binomialTest(200, 200, 0.9950429, > AlternativeHypothesis.TWO_SIDED) == 1.3701357550780435 > In my case, if the expected p-value is 1 (calculated by package in other > language, scipy in this case), the p-value returned could be > 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
