total: 967 total/failed: 29 total/ignored: 91 total/passed: 847 ikev2-32-nat-rw-rekey failed east:output-different road:output-different ikev2-48-nat-cp-start failed road:output-different newoe-20-ipv6 failed east:output-different road:output-different newoe-27-replace-sa-auth-authnull failed road:output-different certoe-12-nat-server failed road:output-different certoe-14-poc-del-slash32 failed east:EXPECTATION certoe-15-poc-east-west failed east:EXPECTATION certoe-17-asymmetric-cert-nat-testing-client-crash failed east:output-different road:output-different certoe-18-pass-then-go-slash32-keyingtries1 failed road:output-different newoe-18-poc-cop-port22-both-reorder failed east:output-different road:output-different dpd-01 failed west:output-different dpd-02 failed east:output-different ikev2-liveness-05 failed west:output-different ikev2-liveness-06 failed east:output-different road:output-different ikev2-liveness-07 failed road:output-different ikev2-child-rekey-06-deadlock failed west:output-different ikev2-child-rekey-09-windows failed road:output-different delete-sa-05 failed west:output-different ikev1-rw-multiple-subnets failed east:output-different ikev1-rw-multiple-subnets-split failed east:output-different road:output-different ikev2-rw-multiple-subnets failed east:output-different ikev2-rw-multiple-subnets-2 failed east:output-different ikev2-rw-multiple-subnets-3 failed east:output-different ikev2-rw-multiple-subnets-4-mismatch failed east:output-different road:output-different ikev2-labeled-ipsec-04-no-label-on-initiator failed east:output-different nss-cert-crl-03-strict failed east:output-different ikev2-mobike-03 failed road:EXPECTATION ikev2-systemrole-03-certs failed west:output-different ikev2-crossing-streams-03-iface failed east:output-different road:output-different
On Thu, 9 Sept 2021 at 20:41, Andrew Cagney <andrew.cag...@gmail.com> wrote: > > > On Sun, 5 Sept 2021 at 11:17, Paul Wouters <p...@nohats.ca> wrote: > >> On Sun, 5 Sep 2021, Andrew Cagney wrote: >> >> > On Sun, 5 Sept 2021 at 09:44, D. Hugh Redelmeier <h...@mimosa.com> >> wrote: >> > I ran the tests suite last night on two machines. It used to be >> that one >> > would expect different results from two runs "due to >> non-determanism", >> > even on the same machine. >> > >> > I got the same results on two machines. That's either good news >> or luck. >> > This is probably the result of hard work: thanks! >> > >> > >> > It's a bit of both, for instance, these two runs had 36 failures: >> > >> https://testing.libreswan.org/?run=v4.5-36-gcb013f4154-main&run=v4.5-35-g6a4754e1fd-main >> > But yes, things are definitely better. >> >> I think Andrew contributed most of this part in the last two months >> actually. But good to >> see indeed! Next up, fixing liveness/dpd and WIP tests :) >> > > Time to move the tests that won't pass to wip? > > >> Paul >> >
_______________________________________________ Swan-dev mailing list Swan-dev@lists.libreswan.org https://lists.libreswan.org/mailman/listinfo/swan-dev