On Thursday, 1 May 2014 at 18:38:15 UTC, w0rp wrote:
On Thursday, 1 May 2014 at 17:04:53 UTC, Xavier Bigand wrote:
Le 01/05/2014 16:01, Atila Neves a écrit :
On Thursday, 1 May 2014 at 11:44:12 UTC, w0rp wrote:
On Thursday, 1 May 2014 at 11:05:55 UTC, Jacob Carlborg
wrote:
On 2014-04-30 23:35, Andrei Alexandrescu wrote:
Agreed. I think we should look into parallelizing all
unittests. --
Andrei
I recommend running the tests in random order as well.
This is a bad idea. Tests could fail only some of the time.
Even if
bugs are missed, I would prefer it if tests did exactly the
same thing
every time.
They _should_ do exactly the same thing every time. Which is
why running
in threads or at random is a great way to enforce that.
Atila
+1
Tests shouldn't be run in a random order all of the time,
perhaps once in a while, manually. Having continuous
integration randomly report build failures is crap. Either you
should always see a build failure, or you shouldn't see it. You
can only test things which are deterministic, at least as far
as what you observe. Running tests in a random order should be
something you do manually, only when you have some ability to
figure out why the tests just failed.
In my experience when a test fails randomly because of ordering,
a while loop on the shell running until failure is enough to
reproduce it in a few seconds. But as others have mentioned,
being able to use a seed to reproduce it exactly is superior.
Atila
