On 2014/05/26 15:13:00, alph wrote:
On 2014/05/26 14:34:04, yurys wrote:
> This still looks fragile. Let's not go this way and change the tests to
avoid
> this non-determinism: let's feed the stack iterator with some mock data
and
> check that the frames it returns match our expectations.
Let me advocate.
Assuming the current test failure probability (caused by small population
size)
is p. By increasing the number of samples 10 times the probability becomes
p^10.
Correct. But p seems to depend on how busy the bot is.
The v8 waterfall suggests the rough value of p is about 1/100. Thus the
patch
makes the probability to be 10^(-20), i.e. one false positive failure per
10^20
runs. Provided the test is run every 3 seconds, we shouldn't see a failure
next
10^13 years. ;-)
I think we may consider it as stable. wdyt?
We can try but I doubt that playing with these numbers while still having
the
test
run for a reasonable time we will make the test stable.
The benefit of this approach is that it covers the same execution paths as
they
run in the user code. The mock data won't let us test e.g. sampler and
inter-thread communications.
Of cause that would test only part of the functionality the test currently
covers.
On the other hand the test seems to be testing too many things where the
most
problematic
one has always been the stack iterator. That's why I'd rather have a stable
test
for
the stack iterator than one covering the real path but flaky. Btw, there are
already some
tests for the inter-thread communication.
https://codereview.chromium.org/306443003/
--
--
v8-dev mailing list
[email protected]
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
