On 2015/05/18 12:10:17, Sven Panne wrote:
On 2015/05/13 07:43:03, dougc wrote:
> [...] Yes this sounds good and improved testing performance. Note the
test
is
still
> slow - more testing has been added. The test takes around 5 seconds
here. If
you
> can give me a realistic time then testing can be scaled to fit? [...]
5 seconds are roughly 2 orders of magnitude to big: If every test we run
during
"make -j32 qc" took that long, it would roughly take 3.5 hours. So the
rule of
thumb is that a unit test should take a few dozen ms at most. Of course
there
is
a tension between good test coverage and testing time, but more exhaustive
tests
should be split off into something separate (we discussed this several
times
already, but I'm not sure what the current state of this is).
Yes, 5 seconds is a bit too long. If you run the test runner with --time, it
will output a list of the longest running tests. You don't want to make this
list :-) Under 100ms would be acceptable.
What you could do for unit testing is: Generate some sane fixed number
(e.g.
1000) of pseudo-random inputs to the function you want to test and check
only
those, probably for various "interesting" range combinations. This is
e.g. the
idea behind Haskell's QuickCheck
(https://hackage.haskell.org/package/QuickCheck), and there are some C++
ports
of this which might be worth considering, this kind of testing problem
comes
up
again and again.
Boundary cases and some systematic tests within reasonable limits would be
fine.
Please don't add pseudo-random tests without a fixed seed; we have that in
the
typer tests and then they fail nondeterministically--even if they catch a
bug,
it may be several CLs too late.
https://codereview.chromium.org/1121573004/
--
--
v8-dev mailing list
[email protected]
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
