Hi Paul,
Good point.
Do you think it's worth adding an env var for running tests with
intermittent failures on the LF infra? In my experience, it is often
hard to reproduce intermittent failures locally so I'm thinking that in
may be useful to have a "penalty box" for flaky tests.
Or I can push a patch to just skip the VRRP tests instead of using
extended tests.
Thanks,
-daw-
On 3/13/2020 1:40 PM, Paul Vinciguerra wrote:
Hi Dave,
Please don't add broken tests to extended tests.
See https://gerrit.fd.io/r/c/vpp/+/24567/12/test/test_bfd.py
Thanks!
On Fri, Mar 13, 2020 at 1:33 PM Dave Wallace <dwallac...@gmail.com
<mailto:dwallac...@gmail.com>> wrote:
Matt,
I will keep an eye on this gerrit and merge it once the verify
jobs have completed.
If there are other tests which fail, are you ok if I add them to
this patch and turn it into a generic 'disable failing tests'
gerrit change?
The other possibility is that this is due to the recent disabling
of the Naginator retry plugin.
I'm going to investigate if this issue may have been masked by
Naginator...
Thanks for your help on keeping the CI operational!
-daw-
On 3/12/2020 12:09 PM, Matthew Smith via Lists.Fd.Io
<http://Lists.Fd.Io> wrote:
Change submitted - https://gerrit.fd.io/r/c/vpp/+/25834.
Verification jobs are running. Hopefully they won't fail :)
-Matt
On Thu, Mar 12, 2020 at 10:22 AM Matthew Smith via Lists.Fd.Io
<http://Lists.Fd.Io> <mgsmith=netgate....@lists.fd.io
<mailto:netgate....@lists.fd.io>> wrote:
I don't have a solution yet, but one observation has popped
up quickly....
In the 2 failed jobs Ray sent links for, one of them had a
test fail which was not related to VRRP. There is a BFD6 test
failure for the NAT change https://gerrit.fd.io/r/c/vpp/+/25462:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2678/archives/
Looking back through a couple of recent failed runs of that
job, there is also a DHCP6 PD test failure for rdma change
https://gerrit.fd.io/r/c/vpp/+/25823:
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2682/archives/
The most obvious common thread between BFD6, DHCP6 and VRRP
to me seems to be that they all maintain state which is
dependent on timers. There could be a more general issue with
timing-sensitive tests. I am going to submit a change which
will prevent the VRRP tests from running temporarily while I
can figure out a proper solution. Based on the above, other
tests may need the same treatment.
-Matt
On Thu, Mar 12, 2020 at 8:57 AM Matthew Smith
<mgsm...@netgate.com <mailto:mgsm...@netgate.com>> wrote:
Hi Ray,
Thanks for bringing it to my attention. I'll look into it.
-Matt
On Thu, Mar 12, 2020 at 8:24 AM Ray Kinsella
<m...@ashroe.eu <mailto:m...@ashroe.eu>> wrote:
Anyone else noticing seeming spurious failures
related to the VRRP plugin's unit tests.
Some examples from un-related commits.
Ray K
nat: timed out session scavenging upgrade
(https://gerrit.fd.io/r/c/vpp/+/25462)
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2678/console.log.gz
==============================================================================
TEST RESULTS:
Scheduled tests: 1138
Executed tests: 1138
Passed tests: 1021
Skipped tests: 112
Failures: 3
Errors: 2
FAILURES AND ERRORS IN TESTS:
Testcase name: IPv4 VRRP Test Case
FAILURE: IPv4 Master VR does not reply for VIP w/
accept mode off
[test_vrrp.TestVRRP4.test_vrrp4_accept_mode_disabled]
FAILURE: IPv4 Master VR preempted by higher
priority backup
[test_vrrp.TestVRRP4.test_vrrp4_master_preempted]
Testcase name: IPv6 VRRP Test Case
FAILURE: IPv6 Master VR preempted by higher
priority backup
[test_vrrp.TestVRRP6.test_vrrp6_master_preempted]
ERROR: IPv6 Backup VR preempts lower priority
master [test_vrrp.TestVRRP6.test_vrrp6_backup_preempts]
Testcase name: Bidirectional Forwarding Detection
(BFD) (IPv6)
ERROR: echo function
[test_bfd.BFD6TestCase.test_echo]
==============================================================================
vlib: startup multi-arch variant configuration
(https://gerrit.fd.io/r/c/vpp/+/25798_
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2675/console.log.gz
==============================================================================
TEST RESULTS:
Scheduled tests: 22
Executed tests: 22
Passed tests: 21
Failures: 1
FAILURES AND ERRORS IN TESTS:
Testcase name: IPv4 VRRP Test Case
FAILURE: IPv4 Master VR preempted by higher
priority backup
[test_vrrp.TestVRRP4.test_vrrp4_master_preempted]
==============================================================================
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#15795): https://lists.fd.io/g/vpp-dev/message/15795
Mute This Topic: https://lists.fd.io/mt/71901798/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
