Thanks Dawid. I read LUCENE-4463, I'm glad that others want this feature too, and I hope that someone will find a solution.
I'll continue to use -Dtests.iters=N and tests.method=X*, in a single JVM. At the moment, I don't think that -Dtests.dups is very useful, not to me at least, as: 1) I often use 'iters' to try and find a seed that breaks my changes 2) If the failure in question is of a concurrent test, tests.dups isn't very efficient This is a great framework. Parallelizing w/ different seeds will make it even better ! Shai On Sun, Dec 9, 2012 at 3:52 PM, Dawid Weiss <dawid.we...@cs.put.poznan.pl>wrote: > > So now I'm even more confused. What is tests.dups good for then? Just > running the same suite multiple times, but not changing the seed? > > Yes. This was originally requested by Mark (I believe) -- we had a > non-deterministic test for which many repetitions were needed. Because > tests.iters cannot be parallelised (they're a sequence of tests under > a single suite) we duplicated the entire suites -- these are > independent and could be distributed across forked jvms. > > > Dawid, can tests.dups use different seeds? Then it'd be really useful > cause it can run these iters in parallel. > > No, not at the moment. I do realize it's a pain and I have it in mind > that it'd be great to have such a possibility but it's conflicting > with the architecture of the runner right now. I don't have a clear > vision how to solve this but I know it's not going to be trivial. > > There is an issue, check out the discussion: > https://issues.apache.org/jira/browse/LUCENE-4463 > > in particular this one: > > https://issues.apache.org/jira/browse/LUCENE-4463?focusedCommentId=13470937&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13470937 > > > -Dtests.iters generate different seeds for each suite/run. It's good if > you > > have a random test and want to test its (or the code's) robustness. > > tests.iters just multiplies every test N times. Whether it gets a > constant seed or a derivative seed depends on annotations and/or > -Dtests.seed override. By default what you say is true but try running > your tests with: > > -Dtests.iters=10 -Dtests.seed=deadbeef:deadf00d > > and you'll see what I mean. > > > -Dtests.dups uses the same seed for every suite, but can run in parallel > JVMs. > > tests.dups is essentially feeding the same class name to the runner > twice. There is one master seed so both classes should run > IDENTICALLY. You can speed up stress testing if you have multiple > cores or if you wish to run the same test over and over (even with a > single JVM) but it's going to be the same test (unless it's > non-deterministic or in other way does not depend on the seed). > > > But I'm sure Dawid has thought about it, and there's some JUnit > limitation > > that does not allow it :). > > This time it's not even JUnit but the code I wrote -- it strictly > follows the principle of a single master seed, it worked great for > other things but I didn't think of repeating the same suite many > times, each time with a different seed and it's hard to integrate it > right now. > > I'm sure there's a way to rewrite the code, I just didn't have the > time to look into it -- lots of stuff piling up, sorry. > > Dawid > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
