One other thing:
There are two sources of randomness. the difficult one is the
interplay between machine speed, disk speed, CPU speed, all the
gremlins that exist with real hardware.
The second is deliberate randomness. You'll see calls to random() (a
static method in the test framework) that deliberately does something
different. For instance it might choose a random locale, or index
varying numbers of docs, something like this to index between 5 and 15
docs.:
int num = random().nextInt(10) + 5
for (int idx = 0; idx < num; ++idx) {
index a doc
}
So when a test fails, in the log you'll see something like "reproduce
with 'ant -Dtestcase=blah -Dtests.seed=2839923749 .....'. Using that
exact line, all the deliberate randomness will be reproduced exactly.
If one of your failing cases fails reliably with a line like above,
please let us know!
FWIW,
On Fri, Aug 24, 2018 at 8:06 AM Charles Sanders <[email protected]> wrote:
>
> Thanks Erick! Excellent info.
>
>
> ________________________________
> From: "Erick Erickson" <[email protected]>
> To: [email protected]
> Sent: Friday, August 24, 2018 11:02:15 AM
> Subject: Re: Tests Fail
>
> Unfortunately yes, we've built up considerable "test debt" that we're
> working on, failures occur intermittently based on random factors and
> are pretty hard to track down. See:
> http://fucit.org/solr-jenkins-reports/failure-report.html
>
> There are a few things I do at this point when a test fails:
> 1> rerun the test a couple of times, and if it passes don't worry. Do
> this by, for instance, 'ant -Dtestcase=MoveReplicaHDFSTest test' from
> the root of your install
> 2> look at the above link to see if your test is "featured"
> 3> check the test itself, if it has the @BadApple notation then it's
> been identified as flakey.
>
> If you can ever figure out _why_ some of these tests randomly fail
> that would be fantastic!
>
> Best,
> Erick
> On Fri, Aug 24, 2018 at 6:44 AM Charles Sanders <[email protected]> wrote:
> >
> > Hello,
> > I'm new at this, so its probably something I've done... or not done. I
> > fetched from master this morning and ran 'ant clean test'. But I'm getting
> > test failures. I'm following the directions found here:
> > https://wiki.apache.org/solr/HowToContribute
> >
> > Is this normal to have test failures from master? I see emails from
> > Jenkins that the build is unstable.
> > [JENKINS] Lucene-Solr-repro - Build # 1295 - Still Unstable
> >
> > Just trying to understand what I should expect. Thanks for helping the new
> > guy.
> > -Charles
> >
> >
> > [junit4] Tests with failures [seed: C83CD1AEBC29ADAF]:
> > [junit4] - org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
> > [junit4] - org.apache.solr.cloud.TestAuthenticationFramework.testBasics
> > [junit4]
> > [junit4]
> > [junit4] JVM J0: 0.89 .. 3088.54 = 3087.65s
> > [junit4] JVM J1: 0.63 .. 3088.48 = 3087.85s
> > [junit4] JVM J2: 0.89 .. 3088.64 = 3087.75s
> > [junit4] Execution time total: 51 minutes 28 seconds
> > [junit4] Tests summary: 829 suites (5 ignored), 4028 tests, 2 errors,
> > 170 ignored (150 assumptions)
> >
> > BUILD FAILED
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
