In case I was not clear: in case of most tests the tests.iters _is_ the
right way to repeat the same test with different seeds. The only difference
is when you would like to randomize static stuff - then tests.iters would
still apply only to tests, not the static test context ( which would remain
the same for all repetitions).
On Wed, Feb 5, 2020, 19:23 Dawid Weiss <dawid.we...@gmail.com> wrote:
>
> Short because I am on a phone only. The first part of the seed.is the
> "static" part that applies to static context - constructors, beforeclass
> etc. The second part would apply to methods. Test.iters is a.property
> recognized by the runner (so it works from the ide etc.) - if your random
> failure is at method level then the second part is relevant.
>
> Gradle build currently does not have "beasting" in the form of different
> static seeds. This would have to be implemented as a series of recursive
> calls with different master seeds...
> Complicated but not impossible.
>
>
>
> On Wed, Feb 5, 2020, 18:49 Michael Sokolov <msoko...@gmail.com> wrote:
>
>> Hi, I've been using gradlew a lot last few days, and first I want to
>> give a huge thanks to those who did the work to make this possible.
>> I'm not naming names since I probably don't know everyone who chipped
>> in, but this is really great. I'm especially loving the `gradlew
>> tasks` and the excellent documentation for the tasks!
>>
>> Now my beasting beef :) I did some beasting and got a bunch of
>> failures, and it looks as if they were all run with the same seed;; eg
>> here's some output from the test showing (I think) two failures with
>> the same seed. I had 43 failures out of 1400, all with the same (first
>> part of) the random seed
>>
>> -
>> org.apache.lucene.search.TestTopFieldCollector.testRandomMaxScoreTermination
>> {seed=[C95A2DAEAA6E2219:1FAD1846BF530502]} (:lucene:core)
>> Test output:
>> /home/
>> ANT.AMAZON.COM/sokolovm/workspace/lucene-solr/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.search.TestTopFieldCollector.txt
>> Reproduce with: gradlew :lucene:core:test --tests
>> "org.apache.lucene.search.TestTopFieldCollector"
>> -Ptests.seed=C95A2DAEAA6E2219 -Ptests.iters=100
>> -Ptests.file.encoding=US-ASCII
>>
>> -
>> org.apache.lucene.search.TestTopFieldCollector.testRandomMaxScoreTermination
>> {seed=[C95A2DAEAA6E2219:11046F532841C712]} (:lucene:core)
>> Test output:
>> /home/
>> ANT.AMAZON.COM/sokolovm/workspace/lucene-solr/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.search.TestTopFieldCollector.txt
>> Reproduce with: gradlew :lucene:core:test --tests
>> "org.apache.lucene.search.TestTopFieldCollector"
>> -Ptests.seed=C95A2DAEAA6E2219 -Ptests.iters=100
>> -Ptests.file.encoding=US-ASCII
>>
>>
>> I tried poking around in randomization.gradle to see if I could verify
>> and/or fix, but I'm not sure how the tests.iters mechanism works - is
>> it built into gradle? Something we added on top?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
