This patch set aims to add the necessary support so that bash written selftests are also able to easily run with a remote traffic generator system, either be it in another netns or one accessible through ssh.
This patch set is a result of the discussion from v1: https://lore.kernel.org/all/[email protected]/ Even though the python infrastructure is already established, some things are easier in bash and it would be a shame to leave behind the bash tests that we already have. This support is based on the requirements described in the tools/testing/selftests/drivers/net/README.rst file. Mainly, the drivers/net selftests should be able to run on a interface specified through the NETIF env variable. On top of that, variables such as REMOTE_TYPE and REMOTE_ARGS define how the remote traffic generator can be accessed. Patch 3/10 parses these env variables and constructs the NETIFS array that bash tests are accustomed to. This is with the intention of enabling already written tests to incur minimal changes. The second patch also defines the TARGETS array which will hold the necessary information about the target on which a specific interface is located. For example, a net.config which looks like below: NETIF=eth0 LOCAL_V4=192.168.1.1 REMOTE_V4=192.168.1.2 REMOTE_TYPE=ssh [email protected] will generate the NETIFS and TARGETS arrays with the following data. NETIFS[p1]="eth0" NETIFS[p2]="eth2" TARGETS[eth0]="local:" TARGETS[eth2]="ssh:[email protected]" The above will be true if on the remote target, the interface which has the 192.168.1.2 address is named eth2. The values held in the TARGETS array will be used by the new 'run_on' helper added in patch 2/10 to know how to run a specific command, on the local system, on another netns or by using ssh. Patch 4/10 updates some helpers to use run_on so that, for example, lib.sh is able to ensure stable MAC addresses even with the remote interface located in another netns. The next 5 patches, 5/10-9/10 update the ethtool_rmon.sh script so that it can work with the kselftest infrastructure and the new NETIF/REMOTE_TYPE etc way of working. Beside updating each ip link or ethtool command to use the run_on helper, the patches also remove any testing done on the remote interface. The last patch adds a new test which checks the standard counters - eth-ctrl, eth-mac and pause - and uses the new infrastructure put in place by the first patches. With this patch set, both tests can be run using a net.config file and run_kselftest.sh as shown below. $ make -C tools/testing/selftests/ TARGETS="drivers/net drivers/net/hw" \ install INSTALL_PATH=/tmp/ksft-net-drv $ cd /tmp/ksft-net-drv/ $ cat > ./drivers/net/net.config <<EOF NETIF=endpmac17 LOCAL_V4=17.0.0.1 REMOTE_V4=17.0.0.2 REMOTE_TYPE=ssh [email protected] EOF $ ./run_kselftest.sh -t drivers/net/hw:ethtool_rmon.sh TAP version 13 1..1 # timeout set to 0 # selftests: drivers/net/hw: ethtool_rmon.sh # TAP version 13 # 1..14 # ok 1 ethtool_rmon.rx-pkts64to64 # ok 2 ethtool_rmon.rx-pkts65to127 # ok 3 ethtool_rmon.rx-pkts128to255 # ok 4 ethtool_rmon.rx-pkts256to511 # ok 5 ethtool_rmon.rx-pkts512to1023 # ok 6 ethtool_rmon.rx-pkts1024to1518 # ok 7 ethtool_rmon.rx-pkts1519to10240 # ok 8 ethtool_rmon.tx-pkts64to64 # ok 9 ethtool_rmon.tx-pkts65to127 # ok 10 ethtool_rmon.tx-pkts128to255 # ok 11 ethtool_rmon.tx-pkts256to511 # ok 12 ethtool_rmon.tx-pkts512to1023 # ok 13 ethtool_rmon.tx-pkts1024to1518 # ok 14 ethtool_rmon.tx-pkts1519to10240 # # Totals: pass:14 fail:0 xfail:0 xpass:0 skip:0 error:0 ok 1 selftests: drivers/net/hw: ethtool_rmon.sh $ ./run_kselftest.sh -t drivers/net/hw:ethtool_std_stats.sh TAP version 13 1..1 # timeout set to 0 # selftests: drivers/net/hw: ethtool_std_stats.sh # TAP version 13 # 1..26 # ok 1 ethtool_std_stats.eth-ctrl-MACControlFramesTransmitted # ok 2 ethtool_std_stats.eth-ctrl-MACControlFramesReceived # ok 3 ethtool_std_stats.eth-mac-FrameCheckSequenceErrors # ok 4 ethtool_std_stats.eth-mac-AlignmentErrors # ok 5 ethtool_std_stats.eth-mac-FramesLostDueToIntMACXmitError # ok 6 ethtool_std_stats.eth-mac-CarrierSenseErrors # SKIP # ok 7 ethtool_std_stats.eth-mac-FramesLostDueToIntMACRcvError # ok 8 ethtool_std_stats.eth-mac-InRangeLengthErrors # SKIP # ok 9 ethtool_std_stats.eth-mac-OutOfRangeLengthField # SKIP # ok 10 ethtool_std_stats.eth-mac-FrameTooLongErrors # SKIP # ok 11 ethtool_std_stats.eth-mac-FramesAbortedDueToXSColls # SKIP # ok 12 ethtool_std_stats.eth-mac-SingleCollisionFrames # SKIP # ok 13 ethtool_std_stats.eth-mac-MultipleCollisionFrames # SKIP # ok 14 ethtool_std_stats.eth-mac-FramesWithDeferredXmissions # SKIP # ok 15 ethtool_std_stats.eth-mac-LateCollisions # SKIP # ok 16 ethtool_std_stats.eth-mac-FramesWithExcessiveDeferral # SKIP # ok 17 ethtool_std_stats.eth-mac-BroadcastFramesXmittedOK # ok 18 ethtool_std_stats.eth-mac-OctetsTransmittedOK # ok 19 ethtool_std_stats.eth-mac-BroadcastFramesReceivedOK # ok 20 ethtool_std_stats.eth-mac-OctetsReceivedOK # ok 21 ethtool_std_stats.eth-mac-FramesTransmittedOK # ok 22 ethtool_std_stats.eth-mac-MulticastFramesXmittedOK # ok 23 ethtool_std_stats.eth-mac-FramesReceivedOK # ok 24 ethtool_std_stats.eth-mac-MulticastFramesReceivedOK # ok 25 ethtool_std_stats.pause-tx_pause_frames # ok 26 ethtool_std_stats.pause-rx_pause_frames # # 10 skipped test(s) detected. Consider enabling relevant config options to improve coverage. # # Totals: pass:16 fail:0 xfail:0 xpass:0 skip:10 error:0 ok 1 selftests: drivers/net/hw: ethtool_std_stats.sh Changes in v4: - 1/10: wrap the lines to max 80 chars - 1/10: replace the if-else with a simple if and return in order to be easier to maintain the 80 chars limit. - 2/10: reworked the helpers so that no global variable is used and information is passed only through parameters - 3/10: reword the entry in README.rst to mention that the different interface names is only a bash restriction and the python infrastructure does not have the same problem. - 3/10: only declare the TARGETS array when necessary - 3/10: add a new flags - DRIVER_TEST_CONFORMANT - that needs to be set by the test - 3/10: rework the check_env() function so that its logic is simpler - 3/10: source drivers/net/net.config only if DRIVER_TEST_CONFORMANT == yes - 3/10: check that NETIF and the remote netif have different names and abort test is not - 4/10: split some lines to 80 chars - 5/10: split one line to 80 chars - 8/10: patch is added in this version so that ethtool_rmon.sh is converted to KTAP output - 9/10: set DRIVER_TEST_CONFORMANT to yes - 9/10: update the commit message to reflect the current output - 9/10: split some more lines to 80 chars - 10/10: move to a the KTAP format output by using ktap_helpers.sh - 10/10: update commit message to reflect the current output - 10/10: use DRIVER_TEST_CONFORMANT Changes in v3: - Change the TARGET variable into CUR_TARGET - Always fallback on running a command locally when either TARGETS is not declared or there is no entry for a specific interface - Swap patches 2 & 3 between them so that the run_cmd used in now patch #2 is defined in patch #2 - Add some more double quoting Changes in v2: - 1/9: Convert jq to the --arg usage form - Patches 2/9-8/9 are new in this version - 9/9: Use the new run_on helper - 9/9: No longer checking that each counter has a 1% tolerance against the target. The only upper limit is UINT32_MAX. - 9/9: Check that both the error counters and the collision related ones are zero since this is the most probable scenario (we don't expect errors with the traffic that we are sending). - 9/9: Removed any checks performed on the remot interface counters since that is being used only as a traffic generator. Ioana Ciornei (10): selftests: forwarding: extend ethtool_std_stats_get with pause statistics selftests: net: add helpers for running a command on other targets selftests: net: extend lib.sh to parse drivers/net/net.config selftests: net: update some helpers to use run_on selftests: drivers: hw: cleanup shellcheck warnings in the rmon test selftests: drivers: hw: test rmon counters only on first interface selftests: drivers: hw: replace counter upper limit with UINT32_MAX in rmon test selftests: drivers: hw: move to KTAP output selftests: drivers: hw: update ethtool_rmon to work with a single local interface selftests: drivers: hw: add test for the ethtool standard counters .../testing/selftests/drivers/net/README.rst | 4 + .../testing/selftests/drivers/net/hw/Makefile | 1 + .../selftests/drivers/net/hw/ethtool_rmon.sh | 82 ++++--- .../drivers/net/hw/ethtool_std_stats.sh | 206 ++++++++++++++++++ tools/testing/selftests/net/forwarding/lib.sh | 131 +++++++++-- tools/testing/selftests/net/lib.sh | 41 +++- 6 files changed, 414 insertions(+), 51 deletions(-) create mode 100755 tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh -- 2.25.1
