Re: [PATCH net-next 7/7] tools: tc-testing: Update README and TODO

[Lucas Bates]

On Wed, Feb 14, 2018 at 2:09 PM, Brenda J. Butler  wrote:
> Signed-off-by: Brenda J. Butler 

Acked-by: Lucas Bates 
> ---
>  tools/testing/selftests/tc-testing/README   | 173 
> +---
>  tools/testing/selftests/tc-testing/TODO.txt |  25 +++-
>  2 files changed, 179 insertions(+), 19 deletions(-)
>
> diff --git a/tools/testing/selftests/tc-testing/README 
> b/tools/testing/selftests/tc-testing/README
> index 970ff294fec8..3a0336782d2d 100644
> --- a/tools/testing/selftests/tc-testing/README
> +++ b/tools/testing/selftests/tc-testing/README
> @@ -14,11 +14,11 @@ REQUIREMENTS
>
>  *  The kernel must have network namespace support
>
> -*   The kernel must have veth support available, as a veth pair is created
> +*  The kernel must have veth support available, as a veth pair is created
> prior to running the tests.
>
> -*  All tc-related features must be built in or available as modules.
> -   To check what is required in current setup run:
> +*  All tc-related features being tested must be built in or available as
> +   modules.  To check what is required in current setup run:
> ./tdc.py -c
>
> Note:
> @@ -44,10 +44,13 @@ using the -p option when running tdc:
>  RUNNING TDC
>  ---
>
> -To use tdc, root privileges are required. tdc will not run otherwise.
> +To use tdc, root privileges are required.  This is because the
> +commands being tested must be run as root.  The code that enforces
> +execution by root uid has been moved into a plugin (see PLUGIN
> +ARCHITECTURE, below).
>
> -All tests are executed inside a network namespace to prevent conflicts
> -within the host.
> +If nsPlugin is linked, all tests are executed inside a network
> +namespace to prevent conflicts within the host.
>
>  Running tdc without any arguments will run all tests. Refer to the section
>  on command line arguments for more information, or run:
> @@ -59,6 +62,33 @@ output captured from the failing test will be printed 
> immediately following
>  the failed test in the TAP output.
>
>
> +OVERVIEW OF TDC EXECUTION
> +-
> +
> +One run of tests is considered a "test suite" (this will be refined in the
> +future).  A test suite has one or more test cases in it.
> +
> +A test case has four stages:
> +
> +  - setup
> +  - execute
> +  - verify
> +  - teardown
> +
> +The setup and teardown stages can run zero or more commands.  The setup
> +stage does some setup if the test needs it.  The teardown stage undoes
> +the setup and returns the system to a "neutral" state so any other test
> +can be run next.  These two stages require any commands run to return
> +success, but do not otherwise verify the results.
> +
> +The execute and verify stages each run one command.  The execute stage
> +tests the return code against one or more acceptable values.  The
> +verify stage checks the return code for success, and also compares
> +the stdout with a regular expression.
> +
> +Each of the commands in any stage will run in a shell instance.
> +
> +
>  USER-DEFINED CONSTANTS
>  --
>
> @@ -70,23 +100,132 @@ executed as part of the test. More will be added as 
> test cases require.
>  Example:
> $TC qdisc add dev $DEV1 ingress
>
> +The NAMES values are used to substitute into the commands in the test cases.
> +
>
>  COMMAND LINE ARGUMENTS
>  --
>
>  Run tdc.py -h to see the full list of available arguments.
>
> --p PATH   Specify the tc executable located at PATH to be used on 
> this
> -  test run
> --cShow the available test case categories in this test file
> --c CATEGORY   Run only tests that belong to CATEGORY
> --f FILE   Read test cases from the JSON file named FILE
> --l [CATEGORY] List all test cases in the JSON file. If CATEGORY is
> -  specified, list test cases matching that category.
> --s ID Show the test case matching ID
> --e ID Execute the test case identified by ID
> --iGenerate unique ID numbers for test cases with no existing
> -  ID number
> +usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]]
> +  [-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v]
> +  [-d DEVICE] [-n NS] [-V]
> +
> +Linux TC unit tests
> +
> +optional arguments:
> +  -h, --helpshow this help message and exit
> +  -p PATH, --path PATH  The full path to the tc executable to use
> +  -v, --verbose Show the commands that are being run
> +  -d DEVICE, --device DEVICE
> +Execute the test case in flower category
> +
> +selection:
> +  select which test cases: files plus directories; filtered by categories
> +  plus testids
> +
> +  -D DIR [DIR ...], --directory DIR [DIR ...]
> +Collect tests from the specified directory(ies)
> +

[PATCH net-next 7/7] tools: tc-testing: Update README and TODO

[Brenda J. Butler]

Signed-off-by: Brenda J. Butler 
---
 tools/testing/selftests/tc-testing/README   | 173 +---
 tools/testing/selftests/tc-testing/TODO.txt |  25 +++-
 2 files changed, 179 insertions(+), 19 deletions(-)

diff --git a/tools/testing/selftests/tc-testing/README 
b/tools/testing/selftests/tc-testing/README
index 970ff294fec8..3a0336782d2d 100644
--- a/tools/testing/selftests/tc-testing/README
+++ b/tools/testing/selftests/tc-testing/README
@@ -14,11 +14,11 @@ REQUIREMENTS
 
 *  The kernel must have network namespace support
 
-*   The kernel must have veth support available, as a veth pair is created
+*  The kernel must have veth support available, as a veth pair is created
prior to running the tests.
 
-*  All tc-related features must be built in or available as modules.
-   To check what is required in current setup run:
+*  All tc-related features being tested must be built in or available as
+   modules.  To check what is required in current setup run:
./tdc.py -c
 
Note:
@@ -44,10 +44,13 @@ using the -p option when running tdc:
 RUNNING TDC
 ---
 
-To use tdc, root privileges are required. tdc will not run otherwise.
+To use tdc, root privileges are required.  This is because the
+commands being tested must be run as root.  The code that enforces
+execution by root uid has been moved into a plugin (see PLUGIN
+ARCHITECTURE, below).
 
-All tests are executed inside a network namespace to prevent conflicts
-within the host.
+If nsPlugin is linked, all tests are executed inside a network
+namespace to prevent conflicts within the host.
 
 Running tdc without any arguments will run all tests. Refer to the section
 on command line arguments for more information, or run:
@@ -59,6 +62,33 @@ output captured from the failing test will be printed 
immediately following
 the failed test in the TAP output.
 
 
+OVERVIEW OF TDC EXECUTION
+-
+
+One run of tests is considered a "test suite" (this will be refined in the
+future).  A test suite has one or more test cases in it.
+
+A test case has four stages:
+
+  - setup
+  - execute
+  - verify
+  - teardown
+
+The setup and teardown stages can run zero or more commands.  The setup
+stage does some setup if the test needs it.  The teardown stage undoes
+the setup and returns the system to a "neutral" state so any other test
+can be run next.  These two stages require any commands run to return
+success, but do not otherwise verify the results.
+
+The execute and verify stages each run one command.  The execute stage
+tests the return code against one or more acceptable values.  The
+verify stage checks the return code for success, and also compares
+the stdout with a regular expression.
+
+Each of the commands in any stage will run in a shell instance.
+
+
 USER-DEFINED CONSTANTS
 --
 
@@ -70,23 +100,132 @@ executed as part of the test. More will be added as test 
cases require.
 Example:
$TC qdisc add dev $DEV1 ingress
 
+The NAMES values are used to substitute into the commands in the test cases.
+
 
 COMMAND LINE ARGUMENTS
 --
 
 Run tdc.py -h to see the full list of available arguments.
 
--p PATH   Specify the tc executable located at PATH to be used on this
-  test run
--cShow the available test case categories in this test file
--c CATEGORY   Run only tests that belong to CATEGORY
--f FILE   Read test cases from the JSON file named FILE
--l [CATEGORY] List all test cases in the JSON file. If CATEGORY is
-  specified, list test cases matching that category.
--s ID Show the test case matching ID
--e ID Execute the test case identified by ID
--iGenerate unique ID numbers for test cases with no existing
-  ID number
+usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]]
+  [-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v]
+  [-d DEVICE] [-n NS] [-V]
+
+Linux TC unit tests
+
+optional arguments:
+  -h, --helpshow this help message and exit
+  -p PATH, --path PATH  The full path to the tc executable to use
+  -v, --verbose Show the commands that are being run
+  -d DEVICE, --device DEVICE
+Execute the test case in flower category
+
+selection:
+  select which test cases: files plus directories; filtered by categories
+  plus testids
+
+  -D DIR [DIR ...], --directory DIR [DIR ...]
+Collect tests from the specified directory(ies)
+(default [tc-tests])
+  -f FILE [FILE ...], --file FILE [FILE ...]
+Run tests from the specified file(s)
+  -c [CATG [CATG ...]], --category [CATG [CATG ...]]
+Run tests only from the specified category/ies, or if
+no category/ies is/are specified, list known
+