+100 very good advices and hints indeed, thanks Clebert! Will do!
Il giorno lun 27 apr 2020 alle ore 18:02 Clebert Suconic <
[email protected]> ha scritto:
> I think this is a general request applying for any test you write on
> any project:
>
> - When you write a new test, please make sure you run the test in a loop
> (Idea has a nice support for that, Edit Test Profile, and make it
> run it 300 times, until fails.. .etc).
> I have been dealing with intermittent failures and most of the time
> the issue was there from day one on the test.
>
> - Do not use Sleep. Unless you really mean to test things like
> expire... and even then, try to make it fast... every second counts
> when you have 1000s tests...
>
> - Use Wait clauses instead of sleep.
>
> - Do not use Assert.assertNull(consumer.receive(long-timeout)); if you
> are doing Assert.assertNull, use consumer.receiveNoWait());
>
> - Limit test output.
> You know that System.out hack you added on the test...
> Things like:
>
> System.out.println("Received message')
> System.out.println("passed through here"); // this one is mine
>
> Please dont do this. Try to limit log output please
>
> In artemis all tests should be using log.debug() as the CI profile
> will disable log.debug(); and only keep log.info().
>
>
>
>
> I have done some major work on the testsuite, even though the
> testsuite is taking 2 hours and 15 (with a variance of 5 minutes +/-),
> it's always passing. (with the caveat of a few retries along the run).
>
> So, I want to keep it that, way, so please be diligent about how you
> write your tests. Avoid flaky tests by running them in a loop before
> you send the PR.
>
> And this is a general request, that I'm pretty sure it's valid for all
> the projects.
>
>
> Thank you!
>
> --
> Clebert Suconic
>
