> So now I'm even more confused. What is tests.dups good for then? Just running > the same suite multiple times, but not changing the seed?
Yes. This was originally requested by Mark (I believe) -- we had a non-deterministic test for which many repetitions were needed. Because tests.iters cannot be parallelised (they're a sequence of tests under a single suite) we duplicated the entire suites -- these are independent and could be distributed across forked jvms. > Dawid, can tests.dups use different seeds? Then it'd be really useful cause > it can run these iters in parallel. No, not at the moment. I do realize it's a pain and I have it in mind that it'd be great to have such a possibility but it's conflicting with the architecture of the runner right now. I don't have a clear vision how to solve this but I know it's not going to be trivial. There is an issue, check out the discussion: https://issues.apache.org/jira/browse/LUCENE-4463 in particular this one: https://issues.apache.org/jira/browse/LUCENE-4463?focusedCommentId=13470937&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470937 > -Dtests.iters generate different seeds for each suite/run. It's good if you > have a random test and want to test its (or the code's) robustness. tests.iters just multiplies every test N times. Whether it gets a constant seed or a derivative seed depends on annotations and/or -Dtests.seed override. By default what you say is true but try running your tests with: -Dtests.iters=10 -Dtests.seed=deadbeef:deadf00d and you'll see what I mean. > -Dtests.dups uses the same seed for every suite, but can run in parallel JVMs. tests.dups is essentially feeding the same class name to the runner twice. There is one master seed so both classes should run IDENTICALLY. You can speed up stress testing if you have multiple cores or if you wish to run the same test over and over (even with a single JVM) but it's going to be the same test (unless it's non-deterministic or in other way does not depend on the seed). > But I'm sure Dawid has thought about it, and there's some JUnit limitation > that does not allow it :). This time it's not even JUnit but the code I wrote -- it strictly follows the principle of a single master seed, it worked great for other things but I didn't think of repeating the same suite many times, each time with a different seed and it's hard to integrate it right now. I'm sure there's a way to rewrite the code, I just didn't have the time to look into it -- lots of stuff piling up, sorry. Dawid --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
