[
https://issues.apache.org/jira/browse/LUCENE-4463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470943#comment-13470943
]
Robert Muir commented on LUCENE-4463:
-------------------------------------
I know its not easy: thats why its a "wish" task :)
I guess I'm just basically listing what could be seen as an expert feature
here, but arguably necessary if you are trying to do randomized tests.
The fact is that things dont always reproduce 100%, and you know this is
definitely a failure in our tests (e.g. the current situation motivated
me to open LUCENE-4460). But really part of random testing is you know, you
dont have to try to write targeted tests but just throw hardware
at the problem (which I'm doing... my office is really hot right now!).
The frustrating part is I think ideally you want to basically treat this whole
randomized test situation like a normal deterministic unit test, you know
like a normal developer would have, so you know you fixed the bug, even if the
test isn't great and doesnt reproduce 100%, you want to know its really
fixed rather than taking blind stabs, waiting to see if all the computers in
your house running full throttle will trip a bug in 24 hours to declare success
:)
So I'm just basically opening this "wish" task to try to think of ways to make
this easier and more efficient.
I'd actually go so far to say the tests.iters is really outdated for lucene's
tests these days (since we have so much class-level "parameterization"
and we should be focusing on this tests.dups (and maybe removing the
tests.iters totally). Maybe thats just particular to us though, but as I
mentioned
above I think we show some real use cases for parameterizing the entire test
class with certain things because it simplifies debugging.
> add support for running the same test method many times
> -------------------------------------------------------
>
> Key: LUCENE-4463
> URL: https://issues.apache.org/jira/browse/LUCENE-4463
> Project: Lucene - Core
> Issue Type: Wish
> Components: general/build
> Reporter: Robert Muir
> Attachments: LUCENE-4463.patch
>
>
> I have a shell script for this, mike has a python script, its annoying :)
> I want to do something like this:
> ant beast -Dtestcase=XXXX -Dtestmethod=YYYY -Diterations=100
> I would be happy with a simple loop that just invokes 'test' somehow: getting
> a fresh new JVM to each iteration is desirable anyway (so you get fresh
> codecs, etc).
> the -Dtests.iters is not really useful for this because it does not allow
> -Dtestmethod and it does not give a fresh jvm.
> bonus points if it can use multiple jvms at the same time though :)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
