[
https://issues.apache.org/jira/browse/FLINK-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316536#comment-15316536
]
Till Rohrmann commented on FLINK-2832:
--------------------------------------
Here is another instance of the failure (this time the
{{testReservoirSamplerWithMultiSourcePartitions1}} test failed):
https://s3.amazonaws.com/archive.travis-ci.org/jobs/134215943/log.txt
The problem seems to be that the testing condition is not deterministic. Every
so and so many test runs the Kolmogorow-Smirnow test simply fails because the
level of significance is not zero.
I would propose to set the level of significance to {{0.001}} instead of
{{0.01}} and to add a retry annotation to the sampler tests. This should harden
the test stability.
> Failing test: RandomSamplerTest.testReservoirSamplerWithReplacement
> -------------------------------------------------------------------
>
> Key: FLINK-2832
> URL: https://issues.apache.org/jira/browse/FLINK-2832
> Project: Flink
> Issue Type: Bug
> Components: Tests
> Affects Versions: 0.10.0
> Reporter: Vasia Kalavri
> Priority: Critical
> Labels: test-stability
> Fix For: 1.0.0
>
>
> Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 19.133 sec
> <<< FAILURE! - in org.apache.flink.api.java.sampling.RandomSamplerTest
> testReservoirSamplerWithReplacement(org.apache.flink.api.java.sampling.RandomSamplerTest)
> Time elapsed: 2.534 sec <<< FAILURE!
> java.lang.AssertionError: KS test result with p value(0.110000), d
> value(0.103090)
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.flink.api.java.sampling.RandomSamplerTest.verifyKSTest(RandomSamplerTest.java:342)
> at
> org.apache.flink.api.java.sampling.RandomSamplerTest.verifyRandomSamplerWithSampleSize(RandomSamplerTest.java:330)
> at
> org.apache.flink.api.java.sampling.RandomSamplerTest.verifyReservoirSamplerWithReplacement(RandomSamplerTest.java:289)
> at
> org.apache.flink.api.java.sampling.RandomSamplerTest.testReservoirSamplerWithReplacement(RandomSamplerTest.java:192)
> Results :
> Failed tests:
>
> RandomSamplerTest.testReservoirSamplerWithReplacement:192->verifyReservoirSamplerWithReplacement:289->verifyRandomSamplerWithSampleSize:330->verifyKSTest:342
> KS test result with p value(0.110000), d value(0.103090)
> Full log [here|https://travis-ci.org/apache/flink/jobs/84120131].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
