This is an automated email from the ASF dual-hosted git repository.
kpvdr pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/qpid-interop-test.git
The following commit(s) were added to refs/heads/main by this push:
new 805bbb8 Further edits to QUICKSTART.md, also updates to
Shim_HOWTO.txt and Test_HOWTO.txt
805bbb8 is described below
commit 805bbb81849f451492e87d206756aaaeaffe7b57
Author: Kim van der Riet <[email protected]>
AuthorDate: Wed Oct 13 15:54:40 2021 -0400
Further edits to QUICKSTART.md, also updates to Shim_HOWTO.txt and
Test_HOWTO.txt
---
QUICKSTART.md | 15 ++++----
README.md | 44 ++++++------------------
docs/Test_HOWTO.txt | 97 +++++++++++++++++++++++++++++-----------------------
docs/users-guide.txt | 8 ++---
4 files changed, 76 insertions(+), 88 deletions(-)
diff --git a/QUICKSTART.md b/QUICKSTART.md
index 7ef82bd..74bdef6 100644
--- a/QUICKSTART.md
+++ b/QUICKSTART.md
@@ -67,29 +67,28 @@ Qpid Python 3 bindings | `python3-qpid-proton` |
`python3-qpid-proton`[^1]
The following are not required, but if installed and present, will be tested:
* Rhea, a JavaScript AMQP client (https://github.com/amqp/rhea.git)
- * dotnet-sdk-5.0
+ * .NET SDK 5.0 (https://docs.microsoft.com/en-us/dotnet/core/install/linux)
Tool | Fedora 34 & CentOS 8 | Ubuntu Focal |
--------------------|-------------------------|------------------------------|
node-js (devel)[^2] | `nodejs-devel` | `libnode-dev` |
-dotnet SDK 5.0[^3] | `dotnet-sdk-5.0` | `aspnetcore-runtime-5.0`[^4] |
+.NET SDK 5.0[^3] | `dotnet-sdk-5.0` | `aspnetcore-runtime-5.0`[^4] |
[^2]: Required to run Rhea Javascript client.
[^3]: Required to run Amqp DotNet Lite client.
[^4]: Must have `packages-microsoft-prod.deb` installed from Microsoft to
install this package.
-In addition, if you wish to run the tests against a broker on the build
machine,
-it will be necessary to have a running broker. One of the following may be
-installed and started against which to run the tests as a local broker:
+In addition, if you wish to run the tests against a local broker on the build
machine,
+it will be necessary to install one. One of the following may be installed and
started:
* Artemis Java broker
(https://activemq.apache.org/components/artemis/download/)
* Qpid Dispatch router, which may be used as a single node, or as part of a
routed configuration. This is available in many distributions as a package.
(https://qpid.apache.org/components/dispatch-router/index.html)
-Tool | Fedora 34 & CentOS 8 | Ubuntu Focal |
----------------------|-------------------------|----------------|
-Qpid Dispatch Router | `qpid-dispatch-router ` | `qdrouterd` |
+Tool | Fedora 34 & CentOS 8 | Ubuntu Focal |
+---------------------|-------------------------|-----------------|
+Qpid Dispatch Router | `qpid-dispatch-router ` | `qdrouterd`[^1] |
Any AMQP 1.0 broker should work.
diff --git a/README.md b/README.md
index cea430f..fb2e5d9 100644
--- a/README.md
+++ b/README.md
@@ -26,13 +26,13 @@ This directory contains the [Qpid Interop
Test](http://qpid.apache.org/component
## Documentation
A quickstart guide for building and using this test suite is contained in
-[QUICKSTART](file:QUICKSTART.md). Detailed documentation for adding tests and
using them are
+[QUICKSTART](QUICKSTART.md). Detailed documentation for adding tests and using
them are
contained in the docs directory.
## Issues
Issues are tracked in the Apache JIRA at
-[https://issues.apache.org/jira/browse/QPIDIT](https://issues.apache.org/jira/browse/QPIDIT)
+[https://issues.apache.org/jira/browse/QPIDIT](https://issues.apache.org/jira/browse/QPIDIT).
## Support
@@ -41,37 +41,15 @@ Support may be obtained from the **qpid-users** mailing list
## Installing and Running
-See the [QUICKSTART](file:QUICKSTART.md) file for building, installing and
running instructions.
+See the [QUICKSTART](QUICKSTART.md) file for building, installing and running
instructions.
-## Writing new shims for a test
+## Writing New Tests
-A detailed description of this process is contained in the docs
-directory. The very short version of this is as follows:
-
-1. Write a pair of client programs using the client API under test. The first
-is a sender which reads the following from the command-line:
-
- `sender <broker address> <amqp type> <JSON string: test values> ...`
-
- and is responsible for sending messages containing the test values each in
a
- single message in the appropriate AMQP type format.
-
- The second client program is a receiver, and must read the following from
the
- command-line
+There are two parts to this:
+1. Writing new tests. This requires adding the test to the Qpid Interop Test
+ Python source.
+2. Writing new shims for the test, one pair (sender and receiver) per client
+ being tested.
- `receiver <broker address> <amqp type> <JSON string: num messages>`
-
- and is responsible for receiving <num messages> messages from the broker
and
- printing the bodies of the received messages appropriately decoded for type
- <amqp type>. The printed output will be a JSON string containing the
identical
- structure to that sent to the send shim, but containing the received
values.
-
-2. Add a subclass for this client in
-`src/py/qpid-interop-test/types/simple_type_test.py`
-derived from class `Shim` and which overrides `NAME`, `ENV` (as needed),
`SHIM_LOC`,
-`SEND` and `RECEIVE`. `SEND` and `RECEIVE` must point to the two clients
written
-in step 1 above.
-
-3. Add an instance of your new shim class to `SHIM_MAP` keyed against its name.
-
-That's it!
+A detailed description of this process is contained in the docs
+directory. See [Shim_HOWTO](docs/Shim_HOWTO.txt) and
[Test_HOWTO](docs/Test_HOWTO.txt).
diff --git a/docs/Test_HOWTO.txt b/docs/Test_HOWTO.txt
index 84e1cbd..17c4d0e 100644
--- a/docs/Test_HOWTO.txt
+++ b/docs/Test_HOWTO.txt
@@ -41,7 +41,7 @@ available, and can handle the necessary data structure.
1. Test name and location
=========================
-Each Test is a single stand-alone Python program located in the
+Each Test is a single stand-alone Python program located in the
src/python/qpid_interop_test directory. The test is named
<test-name>_test.py
@@ -57,20 +57,37 @@ The following modules are available to Tests:
uses of this is to conditionally skip certain tests which fail based
on the broker against which they are running.
-2.2 qit_errors.py
+2.2 qit_common.py
+-----------------
+ Code and classes common to most tests.
+
+ This module includes class QitTestTypeMap which contains empty maps
+ useful for storing test data vs data type for some tests. The class
+ containing this map is subclasses and populated with test data for
+ each test. This implies that the test data is saved as literals
+ within the Test itself (which is current practice). However, using
+ this technique is optional, and test data can be obtained by any
+ useful means, so long as it is easy to access and maintain.
+
+ NOTE: All tests except amqp_complex_types_test use this technique.
+
+ NOTE: Test amqp_complex_types_test stores its test data in JSON
+ files, then uses a code generator to emit test code for each client.
+
+2.3 qit_errors.py
-----------------
Defines error classes for use with tests. Currently this only
defines a single error class (InteropTestError) which accepts
a text error message, but more complex error classes may
be added if needed.
-2.3 qit_jms_types.py
+2.4 qit_jms_types.py
--------------------
Used by JMS tests, this module defines common QpidJms values and
operations.
-2.4 shims.py
-------------
+2.5 qit_shim.py
+---------------
This module defines the interface for calling, sending parameters
to and receiving data from Shims.
@@ -121,33 +138,26 @@ The following modules are available to Tests:
the shim independently from the test using these parameters. This is
especially useful if you need to use a debugger on the shim.
-2.5 test_type_map.py
+2.5 qit_xunit_log.py
--------------------
- This module contains a map useful for storing test data vs data type
- for some tests. This implies that the test data is saved as literals
- within the Test itself (which is current practice). However, using
- this technique is optional, and test data can be obtained by any
- useful means, so long as it is easy to access and maintain.
-
- NOTE: There is some thought going into changing this so that test data
- is stored in some test data file rather than as literals in the Test
- itself. Some changes to the current tests to support this may be proposed
- soon.
-
-NOTE: There is ongoing work to condense common code into modules which
-may be shared between Tests without the need to duplicate.
+ This module allows test results to be stored in xunit format.
3. Command-line options
=======================
-The Test should accept the following command-line options:
-parameter name | action | default | metavar | help
----------------+------------+----------------+--------------+------
---sender | store | localhost:5672 | IP-ADDR:PORT | Node to which
test suite will send messages
---receiver | store | localhost:5672 | IP-ADDR:PORT | Node from which
test suite will receive messages
---no-skip | store_true | | | Do not skip
tests that are excluded by default for reasons of a known bug [1]
---broker-type | store | | BROKER_NAME | Disable test of
broker type (using connection properties) by specifying the broker name, or
"None" [2]
---include-shim | append | | SHIM-NAME | Name of shim to
include [3][4]
---exclude-shim | append | | SHIM-NAME | Name of shim to
exclude [3][5]
+The Tests should accept the following command-line options:
+parameter | action | default | metavar | help
+------------------+------------+------------------+--------------+------
+--sender | store | localhost:5672 | IP-ADDR:PORT | Node to
which test suite will send messages
+--receiver | store | localhost:5672 | IP-ADDR:PORT | Node from
which test suite will receive messages
+--no-skip | store_true | | | Do not skip
tests that are excluded by default for reasons of a known bug [1]
+--broker-type | store | | BROKER_NAME | Disable
test of broker type (using connection properties) by specifying the broker
name, or "None" [2]
+--timeout | store | varies per test | SEC | Timeout for
test in seconds (%d sec). If test is not complete in this time, it will be
terminated
+--include-shim | append | | SHIM-NAME | Name of
shim to include [3][4]
+--exclude-shim | append | | SHIM-NAME | Name of
shim to exclude [3][5]
+--xunit-log | store_true | | | Enable
xUnit logging of test results
+--xunit-log-dir | store | cwd()/xunit_logs | LOG-DIR-PATH | Default
xUnit log directory where xUnit logs are written [xunit_logs dir in current
directory (%s)]
+--description | store | | DESCR | Detailed
description of test, used in xUnit logs
+--broker-topology | store | | DESCR | Detailed
description of broker topology used in test, used in xUnit logs
Notes:
[1] --no-skip: Only applies if you are using the broke name (through
@@ -177,9 +187,9 @@ Tests perform the following tasks:
4.1 Define test cases (and data)
--------------------------------
-This is generally done by subclassing TestTypeMap (see 2.5 above). However,
-any method to obtain a map containing test cases as the index and the test
-data for that case as the value is appropriate.
+This is generally done by subclassing class QitTestTypeMap (see 2.2
+above). However, any method to obtain a map containing test cases as
+the index and the test data for that case as the value is appropriate.
If known test failures exist for certain brokers, then define a
BROKER_SKIP map in which the affected test cases are the index,
@@ -198,20 +208,21 @@ reporting of issues.
4.2 Define your test class
--------------------------
-Derive your test class from unittest.TestCase. Make sure it has a run_test()
-method should receive as parameters the send and receive shim objects being
-used for this test case. This method should perform the following tasks:
+Derive your test class from class QitTestCase. Make sure it has a run_test()
+method, and which should receive as parameters the send and receive shim
+objects being used for this test case. This method should perform the
+following tasks:
* Create a unique queue name for this test;
* Create a Sender object by calling the shim create_sender() method;
* Create a Receiver object by calling the shim create_receiver() method;
* Run the Sender and Receiver by calling the start() method on each object.
- This will cause the Shims to be called through the OS.
+ This will cause the Shims to be called on separate threads through the OS.
* Wait for the Sender and Receiver shims to finish executing by calling
join().
-* Process any returned errors by failing the test
+* Process any returned errors by failing the test.
* If no errors are present, process the returned data and make a decision
- to pass or fail the test. Methods from unittest.TestCase (such as
- assertEqual() or fail() will be needed.
+ to pass or fail the test based on the received data values. Usually these
+ are compared with the sent values to determine correctness.
4.3 Create your test cases
--------------------------
@@ -221,8 +232,8 @@ different and specific, then using the static method will
work better
4.3.1 Static tests methods
--------------------------
-Create one method per test labeled test<test_name> to the test class. See
-https://docs.python.org/2.7/library/unittest.html for details. Each test should
+Create one method per test labeled test<test_name> in the test class. See
+https://docs.python.org/3.8/library/unittest.html for details. Each test should
accomplish a single objective using a single or limited set of similar
messages.
@@ -231,12 +242,14 @@ messages.
Were tests consist of the permutations and combinations of a number of test
shims and test values, it is possible to create the test methods dynamically
which saves a lot of error and repetition. An example may be found in
-amqp_types-test, where the method is create_testcase_class(amqp_type,
shim_product).
+amqp_types-test, where the method which creates the test is
+create_testcase_class(amqp_type, shim_product, timeout).
4.4 Add test options
--------------------
Use argparse.ArgumentParser to add test arguments. See section 3 above for
-recommended options.
+recommended options. Individual tests may also define additional test-specific
+options as needed to control/modify that test.
4.5 Create a shim map
---------------------
diff --git a/docs/users-guide.txt b/docs/users-guide.txt
index 85269a0..6a69439 100644
--- a/docs/users-guide.txt
+++ b/docs/users-guide.txt
@@ -33,7 +33,6 @@ under test. For example, the amqp_types test has shims for
the following
clients:
* AmqpNetLite
* ProtonCpp
-* ProtonPython2
* ProtonPython3
* RheaJS
@@ -116,7 +115,7 @@ environment settings need be made.
$ make
$ sudo make install
----
-
+
*Local install:*
----
$ cd qpid-interop-test
@@ -220,7 +219,7 @@ an issue open to address this limitation.
`--exclude-type`. May be used multiple times to include more
than one type.
|`--exclude-type` |Name of JMS message type to exclude. Cannot be used
together with
`--include-type`. May be used multiple times to exclude more
than one type.
-|===
+|===
.jms-hdrs-props-test
[cols="20%,80%"]
@@ -250,7 +249,7 @@ as follows (first set up the brokers and routers):
+----------+ +----------+ | |
| broker |
+----------+ 9002 +----------+ 5672 | |
-| receiver |<---------| dispatch |<--------| |
+| receiver |<---------| dispatch |<--------| |
| shim | | router 2 | | |
+----------+ +----------+ +--------+
----
@@ -279,4 +278,3 @@ will be generated at the conclusion of the test in the
`xunit_logs` directory. T
using the `--xunit-log-dir` command-line option. Log file management is the
responsibility of the user, and if
xunit logging is used regularly, these files will accumulate, possibly
exhausting the available disk space. NOTE
that if a test is interrupted (by using ctl+c, for example), then the log
files will not be generated.)
-
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
