For what it's worth, calling "ant test-help" from the top-level of the project gives a lot of information about flags, their semantics, filtering, etc.
Dawid On Mon, Aug 27, 2018 at 7:38 PM Jason Gerlowski <[email protected]> wrote: > > You can also tell the test framework to temporarily skip the bad apple > tests with the flag "-Dtests.badapples=false", which is sometimes > useful. Just an FYI. > > Jason > On Fri, Aug 24, 2018 at 11:14 AM Erick Erickson <[email protected]> > wrote: > > > > One other thing: > > There are two sources of randomness. the difficult one is the > > interplay between machine speed, disk speed, CPU speed, all the > > gremlins that exist with real hardware. > > > > The second is deliberate randomness. You'll see calls to random() (a > > static method in the test framework) that deliberately does something > > different. For instance it might choose a random locale, or index > > varying numbers of docs, something like this to index between 5 and 15 > > docs.: > > > > int num = random().nextInt(10) + 5 > > for (int idx = 0; idx < num; ++idx) { > > index a doc > > } > > > > So when a test fails, in the log you'll see something like "reproduce > > with 'ant -Dtestcase=blah -Dtests.seed=2839923749 .....'. Using that > > exact line, all the deliberate randomness will be reproduced exactly. > > If one of your failing cases fails reliably with a line like above, > > please let us know! > > > > FWIW, > > On Fri, Aug 24, 2018 at 8:06 AM Charles Sanders <[email protected]> wrote: > > > > > > Thanks Erick! Excellent info. > > > > > > > > > ________________________________ > > > From: "Erick Erickson" <[email protected]> > > > To: [email protected] > > > Sent: Friday, August 24, 2018 11:02:15 AM > > > Subject: Re: Tests Fail > > > > > > Unfortunately yes, we've built up considerable "test debt" that we're > > > working on, failures occur intermittently based on random factors and > > > are pretty hard to track down. See: > > > http://fucit.org/solr-jenkins-reports/failure-report.html > > > > > > There are a few things I do at this point when a test fails: > > > 1> rerun the test a couple of times, and if it passes don't worry. Do > > > this by, for instance, 'ant -Dtestcase=MoveReplicaHDFSTest test' from > > > the root of your install > > > 2> look at the above link to see if your test is "featured" > > > 3> check the test itself, if it has the @BadApple notation then it's > > > been identified as flakey. > > > > > > If you can ever figure out _why_ some of these tests randomly fail > > > that would be fantastic! > > > > > > Best, > > > Erick > > > On Fri, Aug 24, 2018 at 6:44 AM Charles Sanders <[email protected]> > > > wrote: > > > > > > > > Hello, > > > > I'm new at this, so its probably something I've done... or not done. I > > > > fetched from master this morning and ran 'ant clean test'. But I'm > > > > getting test failures. I'm following the directions found here: > > > > https://wiki.apache.org/solr/HowToContribute > > > > > > > > Is this normal to have test failures from master? I see emails from > > > > Jenkins that the build is unstable. > > > > [JENKINS] Lucene-Solr-repro - Build # 1295 - Still Unstable > > > > > > > > Just trying to understand what I should expect. Thanks for helping the > > > > new guy. > > > > -Charles > > > > > > > > > > > > [junit4] Tests with failures [seed: C83CD1AEBC29ADAF]: > > > > [junit4] - org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove > > > > [junit4] - > > > > org.apache.solr.cloud.TestAuthenticationFramework.testBasics > > > > [junit4] > > > > [junit4] > > > > [junit4] JVM J0: 0.89 .. 3088.54 = 3087.65s > > > > [junit4] JVM J1: 0.63 .. 3088.48 = 3087.85s > > > > [junit4] JVM J2: 0.89 .. 3088.64 = 3087.75s > > > > [junit4] Execution time total: 51 minutes 28 seconds > > > > [junit4] Tests summary: 829 suites (5 ignored), 4028 tests, 2 > > > > errors, 170 ignored (150 assumptions) > > > > > > > > BUILD FAILED > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
