tdc - Linux Traffic Control (tc) unit testing suite
|
|
Author: Lucas Bates - lucasb@mojatatu.com
|
|
tdc is a Python script to load tc unit tests from a separate JSON file and
|
execute them inside a network namespace dedicated to the task.
|
|
|
REQUIREMENTS
|
------------
|
|
* Minimum Python version of 3.4. Earlier 3.X versions may work but are not
|
guaranteed.
|
|
* The kernel must have network namespace support
|
|
* The kernel must have veth support available, as a veth pair is created
|
prior to running the tests.
|
|
* The kernel must have the appropriate infrastructure enabled to run all tdc
|
unit tests. See the config file in this directory for minimum required
|
features. As new tests will be added, config options list will be updated.
|
|
* All tc-related features being tested must be built in or available as
|
modules. To check what is required in current setup run:
|
./tdc.py -c
|
|
Note:
|
In the current release, tdc run will abort due to a failure in setup or
|
teardown commands - which includes not being able to run a test simply
|
because the kernel did not support a specific feature. (This will be
|
handled in a future version - the current workaround is to run the tests
|
on specific test categories that your kernel supports)
|
|
|
BEFORE YOU RUN
|
--------------
|
|
The path to the tc executable that will be most commonly tested can be defined
|
in the tdc_config.py file. Find the 'TC' entry in the NAMES dictionary and
|
define the path.
|
|
If you need to test a different tc executable on the fly, you can do so by
|
using the -p option when running tdc:
|
./tdc.py -p /path/to/tc
|
|
|
RUNNING TDC
|
-----------
|
|
To use tdc, root privileges are required. This is because the
|
commands being tested must be run as root. The code that enforces
|
execution by root uid has been moved into a plugin (see PLUGIN
|
ARCHITECTURE, below).
|
|
If nsPlugin is linked, all tests are executed inside a network
|
namespace to prevent conflicts within the host.
|
|
Running tdc without any arguments will run all tests. Refer to the section
|
on command line arguments for more information, or run:
|
./tdc.py -h
|
|
tdc will list the test names as they are being run, and print a summary in
|
TAP (Test Anything Protocol) format when they are done. If tests fail,
|
output captured from the failing test will be printed immediately following
|
the failed test in the TAP output.
|
|
|
OVERVIEW OF TDC EXECUTION
|
-------------------------
|
|
One run of tests is considered a "test suite" (this will be refined in the
|
future). A test suite has one or more test cases in it.
|
|
A test case has four stages:
|
|
- setup
|
- execute
|
- verify
|
- teardown
|
|
The setup and teardown stages can run zero or more commands. The setup
|
stage does some setup if the test needs it. The teardown stage undoes
|
the setup and returns the system to a "neutral" state so any other test
|
can be run next. These two stages require any commands run to return
|
success, but do not otherwise verify the results.
|
|
The execute and verify stages each run one command. The execute stage
|
tests the return code against one or more acceptable values. The
|
verify stage checks the return code for success, and also compares
|
the stdout with a regular expression.
|
|
Each of the commands in any stage will run in a shell instance.
|
|
|
USER-DEFINED CONSTANTS
|
----------------------
|
|
The tdc_config.py file contains multiple values that can be altered to suit
|
your needs. Any value in the NAMES dictionary can be altered without affecting
|
the tests to be run. These values are used in the tc commands that will be
|
executed as part of the test. More will be added as test cases require.
|
|
Example:
|
$TC qdisc add dev $DEV1 ingress
|
|
The NAMES values are used to substitute into the commands in the test cases.
|
|
|
COMMAND LINE ARGUMENTS
|
----------------------
|
|
Run tdc.py -h to see the full list of available arguments.
|
|
usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]]
|
[-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v] [-N]
|
[-d DEVICE] [-P] [-n] [-V]
|
|
Linux TC unit tests
|
|
optional arguments:
|
-h, --help show this help message and exit
|
-p PATH, --path PATH The full path to the tc executable to use
|
-v, --verbose Show the commands that are being run
|
-N, --notap Suppress tap results for command under test
|
-d DEVICE, --device DEVICE
|
Execute the test case in flower category
|
-P, --pause Pause execution just before post-suite stage
|
|
selection:
|
select which test cases: files plus directories; filtered by categories
|
plus testids
|
|
-D DIR [DIR ...], --directory DIR [DIR ...]
|
Collect tests from the specified directory(ies)
|
(default [tc-tests])
|
-f FILE [FILE ...], --file FILE [FILE ...]
|
Run tests from the specified file(s)
|
-c [CATG [CATG ...]], --category [CATG [CATG ...]]
|
Run tests only from the specified category/ies, or if
|
no category/ies is/are specified, list known
|
categories.
|
-e ID [ID ...], --execute ID [ID ...]
|
Execute the specified test cases with specified IDs
|
|
action:
|
select action to perform on selected test cases
|
|
-l, --list List all test cases, or those only within the
|
specified category
|
-s, --show Display the selected test cases
|
-i, --id Generate ID numbers for new test cases
|
|
netns:
|
options for nsPlugin (run commands in net namespace)
|
|
-n, --namespace
|
Run commands in namespace as specified in tdc_config.py
|
|
valgrind:
|
options for valgrindPlugin (run command under test under Valgrind)
|
|
-V, --valgrind Run commands under valgrind
|
|
|
PLUGIN ARCHITECTURE
|
-------------------
|
|
There is now a plugin architecture, and some of the functionality that
|
was in the tdc.py script has been moved into the plugins.
|
|
The plugins are in the directory plugin-lib. The are executed from
|
directory plugins. Put symbolic links from plugins to plugin-lib,
|
and name them according to the order you want them to run.
|
|
Example:
|
|
bjb@bee:~/work/tc-testing$ ls -l plugins
|
total 4
|
lrwxrwxrwx 1 bjb bjb 27 Oct 4 16:12 10-rootPlugin.py -> ../plugin-lib/rootPlugin.py
|
lrwxrwxrwx 1 bjb bjb 25 Oct 12 17:55 20-nsPlugin.py -> ../plugin-lib/nsPlugin.py
|
-rwxr-xr-x 1 bjb bjb 0 Sep 29 15:56 __init__.py
|
|
The plugins are a subclass of TdcPlugin, defined in TdcPlugin.py and
|
must be called "SubPlugin" so tdc can find them. They are
|
distinguished from each other in the python program by their module
|
name.
|
|
This base class supplies "hooks" to run extra functions. These hooks are as follows:
|
|
pre- and post-suite
|
pre- and post-case
|
pre- and post-execute stage
|
adjust-command (runs in all stages and receives the stage name)
|
|
The pre-suite hook receives the number of tests and an array of test ids.
|
This allows you to dump out the list of skipped tests in the event of a
|
failure during setup or teardown stage.
|
|
The pre-case hook receives the ordinal number and test id of the current test.
|
|
The adjust-command hook receives the stage id (see list below) and the
|
full command to be executed. This allows for last-minute adjustment
|
of the command.
|
|
The stages are identified by the following strings:
|
|
- pre (pre-suite)
|
- setup
|
- command
|
- verify
|
- teardown
|
- post (post-suite)
|
|
|
To write a plugin, you need to inherit from TdcPlugin in
|
TdcPlugin.py. To use the plugin, you have to put the
|
implementation file in plugin-lib, and add a symbolic link to it from
|
plugins. It will be detected at run time and invoked at the
|
appropriate times. There are a few examples in the plugin-lib
|
directory:
|
|
- rootPlugin.py:
|
implements the enforcement of running as root
|
- nsPlugin.py:
|
sets up a network namespace and runs all commands in that namespace
|
- valgrindPlugin.py
|
runs each command in the execute stage under valgrind,
|
and checks for leaks.
|
This plugin will output an extra test for each test in the test file,
|
one is the existing output as to whether the test passed or failed,
|
and the other is a test whether the command leaked memory or not.
|
(This one is a preliminary version, it may not work quite right yet,
|
but the overall template is there and it should only need tweaks.)
|
|
|
ACKNOWLEDGEMENTS
|
----------------
|
|
Thanks to:
|
|
Jamal Hadi Salim, for providing valuable test cases
|
Keara Leibovitz, who wrote the CLI test driver that I used as a base for the
|
first version of the tc testing suite. This work was presented at
|
Netdev 1.2 Tokyo in October 2016.
|
Samir Hussain, for providing help while I dove into Python for the first time
|
and being a second eye for this code.
|
