On Mon, 2015-11-30 at 16:49 +0000, Ian Jackson wrote:
> Fold `guest-localmigrate.2' into `guest-localmigrate/x10' and move
> `guest-start.2' to after `guest-start.repeat' (reversing the contents
> of the latter so that the start comes before the stop).
> (guest-start.2 is still necessary because the start/stop test leaves
> the guest stopped, whereas the subsequent destroy test ought happen
> with the guest running.)
>
> This change will allow the heisenbug compensator to see more of these
> failures as the same failures.
>
> The overall effect includes a reduction of the number of localhost
> migrations from 11 to 10, but this is better than leaving a misleading
> testid containing the string `x10' (or changing the testid).
>
> It is best to fold this way, keeping the testid of the step which
> previously had most of the regressions, because: the alternative,
> keeping the testid of the low-repetition step, would allow osstest to
> use previous lucky passes of the low-repetition step to justify
> current failures of the now-high-repetition step.
>
> To check that the effect of the patch is as intended, I ran a before
> and after run with OSSTEST_SIMULATE=1, and (a) collected and sedded
> and diffed the sg-run-job transcripts and (b) looked in the db.
>
> I also ran a real test (65261 in the Xen Project test lab) with a very
> similar version, which passed, and will re-run that before pushing.
>
> (a):
>
> c&p transcripts from mg-execute-flight email reports
> perl -i~ -pe 's/\b(38371|38370|65261|38395|38397)\b/FLIGHT/; s/^2015-
> 11-\d\d \S+ /TIME /' [tu]
> diff -u [tu] >v
> grep starting v
>
> =>
>
> TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-build-
> check build-check(1)
> TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-
> saverestore host debian guest-saverestore.2
> -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-
> localmigrate host debian guest-localmigrate.2
> TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-
> localmigrate x10 host debian guest-localmigrate/x10
> TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-
> stop host debian guest-stop
> +TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-
> repeat-test 10 ts-guest-start host debian {;} ts-guest-stop host debian
> guest-start/debian.repeat
> TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-
> start host debian guest-start.2
> -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-
> repeat-test 10 ts-guest-stop host debian {;} ts-guest-start host debian
> guest-start/debian.repeat
> TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-
> destroy host debian guest-destroy
> TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-leak-
> check check host leak-check/check
> -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-logs-
> capture host capture-logs(24)
> +TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-logs-
> capture host capture-logs(23)
>
> (b)
>
> osstestdb=> select * from (select job,stepno,step,status,testid from
> steps where flight=38370) before full outer join (select
> job,stepno,step,status,testid from steps where flight=38400) after using
> (testid) order by coalesce(before.stepno, after.stepno);
> testid | job | stepno
> | step | status | job | stepno
> | step | status
> ---------------------------+--------------------+--------+---------------
> ---------------+--------+--------------------+--------+----------------
> --------------+--------
> build-check(1) | test-amd64-i386-xl | 1 | ts-build-
> check | pass | test-amd64-i386-xl | 1 | ts-build-
> check | pass
> hosts-allocate | test-amd64-i386-xl | 2 | ts-hosts-
> allocate | pass | test-amd64-i386-xl | 2 | ts-hosts-
> allocate | pass
> host-install(3) | test-amd64-i386-xl | 3 | ts-host-
> install-twice | pass | test-amd64-i386-xl | 3 | ts-host-
> install-twice | pass
> host-ping-check-native | test-amd64-i386-xl | 4 | ts-host-ping-
> check | pass | test-amd64-i386-xl | 4 | ts-host-ping-
> check | pass
> xen-install | test-amd64-i386-xl | 5 | ts-xen-
> install | pass | test-amd64-i386-xl | 5 | ts-xen-
> install | pass
> xen-boot | test-amd64-i386-xl | 6 | ts-host-
> reboot | pass | test-amd64-i386-xl | 6 | ts-host-
> reboot | pass
> host-ping-check-xen | test-amd64-i386-xl | 7 | ts-host-ping-
> check | pass | test-amd64-i386-xl | 7 | ts-host-ping-
> check | pass
> leak-check/basis(8) | test-amd64-i386-xl | 8 | ts-leak-
> check | pass | test-amd64-i386-xl | 8 | ts-leak-
> check | pass
> debian-install | test-amd64-i386-xl | 9 | ts-debian-
> install | pass | test-amd64-i386-xl | 9 | ts-debian-
> install | pass
> debian-fixup | test-amd64-i386-xl | 10 | ts-debian-
> fixup | pass | test-amd64-i386-xl | 10 | ts-debian-
> fixup | pass
> guest-start | test-amd64-i386-xl | 11 | ts-guest-
> start | pass | test-amd64-i386-xl | 11 | ts-guest-
> start | pass
> migrate-support-check | test-amd64-i386-xl | 12 | ts-migrate-
> support-check | pass | test-amd64-i386-xl | 12 | ts-migrate-
> support-check | pass
> saverestore-support-check | test-amd64-i386-xl | 13 | ts-
> saverestore-support-check | pass | test-amd64-i386-xl | 13 | ts-
> saverestore-support-check | pass
> guest-saverestore | test-amd64-i386-xl | 14 | ts-guest-
> saverestore | pass | test-amd64-i386-xl | 14 | ts-guest-
> saverestore | pass
> guest-localmigrate | test-amd64-i386-xl | 15 | ts-guest-
> localmigrate | pass | test-amd64-i386-xl | 15 | ts-guest-
> localmigrate | pass
> guest-saverestore.2 | test-amd64-i386-xl | 16 | ts-guest-
> saverestore | pass | test-amd64-i386-xl | 16 | ts-guest-
> saverestore | pass
> guest-localmigrate.2 | test-amd64-i386-xl | 17 | ts-guest-
> localmigrate |
> pass | | | |
> guest-localmigrate/x10 | test-amd64-i386-xl | 18 | ts-guest-
> localmigrate | pass | test-amd64-i386-xl | 17 | ts-guest-
> localmigrate | pass
> guest-stop | test-amd64-i386-xl | 19 | ts-guest-
> stop | pass | test-amd64-i386-xl | 18 | ts-guest-
> stop | pass
> guest-start.2 | test-amd64-i386-xl | 20 | ts-guest-
> start | pass | test-amd64-i386-xl | 20 | ts-guest-
> start | pass
> guest-start/debian.repeat | test-amd64-i386-xl | 21 | ts-repeat-
> test | pass | test-amd64-i386-xl | 19 | ts-repeat-
> test | pass
> guest-destroy | test-amd64-i386-xl | 22 | ts-guest-
> destroy | pass | test-amd64-i386-xl | 21 | ts-guest-
> destroy | pass
> leak-check/check | test-amd64-i386-xl | 23 | ts-leak-
> check | pass | test-amd64-i386-xl | 22 | ts-leak-
> check | pass
> capture-
> logs(23) | | |
> | | test-amd64-i386-xl | 23 | ts-logs-
> capture | pass
> capture-logs(24) | test-amd64-i386-xl | 24 | ts-logs-
> capture |
> pass | | | |
> (25 rows)
>
> osstestdb=>
>
> Signed-off-by: Ian Jackson <ian.jack...@eu.citrix.com>
Acked-by: Ian Campbell <ian.campb...@citrix.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
