** Description changed: [Impact] - Test result on Jammy reports there are 5 failed test cases in this fcnal-test.sh test: + The net/fcnal-test.sh on F/J won't return a non-zero value even with + some sub test cases failed, here is an example on Jammy: # Tests passed: 857 - # Tests failed: 5 + # Tests failed: 5 ok 1 selftests: net: fcnal-test.sh - However the test didn't return a non-zero value to mark it as failed, making it difficult to spot these failures: + Therefore it's marked as PASSED on our report and making it difficult + to spot these failures: # With VRF # SYSCTL: net.ipv4.raw_l3mdev_accept=1 - # TEST: ping local, VRF bind - VRF IP [FAIL] - # TEST: ping local, device bind - ns-A IP [FAIL] + # TEST: ping local, VRF bind - VRF IP [FAIL] + # TEST: ping local, device bind - ns-A IP [FAIL] # SYSCTL: net.ipv4.ping_group_range=0 2147483647 # SYSCTL: net.ipv4.raw_l3mdev_accept=1 - # TEST: ping local, VRF bind - VRF IP [FAIL] - # TEST: ping local, device bind - ns-A IP [FAIL] + # TEST: ping local, VRF bind - VRF IP [FAIL] + # TEST: ping local, device bind - ns-A IP [FAIL] # With VRF # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # SYSCTL: net.ipv4.ping_group_range=0 2147483647 # SYSCTL: net.ipv4.raw_l3mdev_accept=1 - # TEST: ping out, vrf device+address bind - ns-B IPv6 LLA [FAIL] + # TEST: ping out, vrf device+address bind - ns-B IPv6 LLA [FAIL] [Fix] * 0f8a3b48f9 selftests: net/fcnal-test.sh: add exit code + This patch can be cherry-picked into Focal and Jammy kernel. + We don't have this test in Bionic, and this patch has already landed + on Kinetic and OEM-5.17 + [Test] - Run the patched test, the return value will be 1 whenever the number of failed test cases is not 0. + Run the patched test, the return value will be 1 whenever the number + of failed test cases is not 0. [Where problems could occur] - This will help us to catch failures in our test report, we're expected to see new failures in Jammy. With bug 2006391 fixed, we will see failures reported in F/K as well. + Change limited to testing tools, but we're expected to see new failures + in Jammy. With bug 2006391 fixed later on, we will see failures + reported in F/K as well.
-- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2006692 Title: net:fcnal-test.sh didn't return a non-zero value even with some sub- tests failed Status in ubuntu-kernel-tests: In Progress Status in linux package in Ubuntu: Fix Released Status in linux source package in Focal: In Progress Status in linux source package in Jammy: In Progress Status in linux source package in Kinetic: Fix Released Bug description: [Impact] The net/fcnal-test.sh on F/J won't return a non-zero value even with some sub test cases failed, here is an example on Jammy: # Tests passed: 857 # Tests failed: 5 ok 1 selftests: net: fcnal-test.sh Therefore it's marked as PASSED on our report and making it difficult to spot these failures: # With VRF # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # SYSCTL: net.ipv4.ping_group_range=0 2147483647 # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # TEST: ping local, VRF bind - VRF IP [FAIL] # TEST: ping local, device bind - ns-A IP [FAIL] # With VRF # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # SYSCTL: net.ipv4.ping_group_range=0 2147483647 # SYSCTL: net.ipv4.raw_l3mdev_accept=1 # TEST: ping out, vrf device+address bind - ns-B IPv6 LLA [FAIL] [Fix] * 0f8a3b48f9 selftests: net/fcnal-test.sh: add exit code This patch can be cherry-picked into Focal and Jammy kernel. We don't have this test in Bionic, and this patch has already landed on Kinetic and OEM-5.17 [Test] Run the patched test, the return value will be 1 whenever the number of failed test cases is not 0. [Where problems could occur] Change limited to testing tools, but we're expected to see new failures in Jammy. With bug 2006391 fixed later on, we will see failures reported in F/K as well. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2006692/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
