[
https://issues.apache.org/jira/browse/MATH-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119131#comment-17119131
]
Gilles Sadowski commented on MATH-1535:
---------------------------------------
The issue is in the
[fixTies|https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/main/java/org/apache/commons/math4/stat/inference/KolmogorovSmirnovTest.java;h=e85cc0986a3ccc78aa8e17d98c2c73b854741459;hb=HEAD#l1082]
method.
I think that it is caused by the hard-coded maximal count (10) and/or the
allowed
[jitter|https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/main/java/org/apache/commons/math4/stat/inference/KolmogorovSmirnovTest.java;h=e85cc0986a3ccc78aa8e17d98c2c73b854741459;hb=HEAD#l1139]
range.
It is conceivable that unexpectedly "ugly data" would hit the safeguards that
assume "nice" data. In the latter case, one would rarely (?) resort to the
above workaround, so I'd guess that any hack would be fine. For example, we
could increase the jitter range and redo the loop for as long as there are ties
(perhaps at the risk of changing the data too much).
What do you think?
> MathInternalError in KolmogorovSmirnovTest in case of many ties
> ---------------------------------------------------------------
>
> Key: MATH-1535
> URL: https://issues.apache.org/jira/browse/MATH-1535
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.6.1
> Environment: commons-math 3.6.1, oracle jdk11 for windows
> Reporter: Steffen Herbold
> Priority: Major
>
> I encounter a math internal error with some very ugly data that has lots of
> ties. The code below triggers the exception. I could try to build in a
> detection in my code that identifies this strange case where the generated
> data has many ties, to avoid this. But I guess the MathInternalError in
> commons-math should still be avoided.
>
> {code:java}
> // works
> double[] sample1 = new double[] {0.8767630865438496, 0.9998809418147052,
> 0.9999999715463531, 0.9999985849345421};
> double[] sample2 = new double[] {1.0, 1.0, 1.0, 1.0};
> ksTest.kolmogorovSmirnovTest(sample1, sample2);
> // fails with illegal state
> double[] sample3 = new double[] {0.8767630865438496, 0.9998809418147052,
> 0.9999999715463531, 0.9999985849345421, 0.973584315883326,
> 0.9999999875782982, 0.999999999999994, 0.9999999999908233, 1.0,
> 0.9999999890925574, 0.9999998345734327, 0.9999999350772448,
> 0.999999999999426, 0.9999147040688201, 0.9999999999999922, 1.0, 1.0,
> 0.9919050954798272, 0.8649014770687263, 0.9990869497973084,
> 0.9993222540990464, 0.999999999998189, 0.9999999999999365,
> 0.9790934801762917, 0.9999578695006303, 0.9999999999999998,
> 0.999999999996166, 0.9999999999995546, 0.9999999999908036, 0.99999999999744,
> 0.9999998802655555, 0.9079334221214075, 0.9794398308007372,
> 0.9999044231134367, 0.9999999999999813, 0.9999957841707683,
> 0.9277678892094009, 0.999948269893843, 0.9999999886132888,
> 0.9999998909699096, 0.9999099536620326, 0.9999999962217623,
> 0.9138936987350447, 0.9999999999779976, 0.999999999998822, 0.999979247207911,
> 0.9926904388316407, 1.0, 0.9999999999998814, 1.0, 0.9892505696426215,
> 0.9999996514123723, 0.9999999999999429, 0.9999999995399116,
> 0.999999999948221, 0.7358264887843119, 0.9999999994098534, 1.0,
> 0.9999986456748472, 1.0, 0.9999999999921501, 0.9999999999999996,
> 0.9999999999999944, 0.9473070068606853, 0.9993714060209042,
> 0.9999999409098718, 0.9999999592791519, 0.9999999999999805};
> double[] sample4 = new double[] {1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
> 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
> 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
> 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
> 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
> 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
> 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
> 1.0};
> ksTest.kolmogorovSmirnovTest(sample3, sample4);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
