[
https://issues.apache.org/jira/browse/QPIDIT-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464023#comment-16464023
]
Kim van der Riet commented on QPIDIT-93:
----------------------------------------
A more comprehensive xUnit XML schema is available here
[https://github.com/windyroad/JUnit-Schema]
and which includes a <{{properties>}} element containing 0..n <{{property>}}
elements, each of which contains a name/value pair of attributes. This can be
used to add additional information about the test in a compliant way as follows:
{noformat}
<testsuite ...>
<properties>
<property name="..." value="..."/>
<property name="..." value="..."/>
<property name="..." value="..."/>
...
</properties>
<testcase ... />
<testcase ... />
<testcase ... />
...
</testsuite>{noformat}
The question is, what information should be included?
A minimum list should be:
* test machine identity
* test executable path/name
* parameters to test executable
* broker topology (single-node, Sender->B1->B2->B3->Receiver, etc)
* broker ip address(es)
* broker configuration(s) - how to achieve this?
In addition, nice-to-have might include:
* Date/timestamp/duration for test
* Additional test description as passed on command-line
* Broker connection properties (if necessary, for both sender and receiver if
different)
Each {{<testcase>}} element supports the following attributes:
* name
* classname
* time
but I would like to add the following out-of-band attributes:
* receiver-client
* sender-client
* type
* duration of individual testcase
* any other specialized parameter used in individual tests:
* subtype (amqp_complex_types_test)
* size (amqp_large_content_test)
* headers (jms_hdrs_props_test)
* properties (jms_hdrs_props_test)
Ideas?
> Optionally produce xUnit XML report with test results
> -----------------------------------------------------
>
> Key: QPIDIT-93
> URL: https://issues.apache.org/jira/browse/QPIDIT-93
> Project: Apache QPID Interoperability Test Suite
> Issue Type: Improvement
> Affects Versions: 0.1.0
> Reporter: Jiri Daněk
> Assignee: Kim van der Riet
> Priority: Minor
> Fix For: 0.2.0
>
> Attachments: amqp_types_test.2018-01-26T09-24-24.xml,
> jms_hdrs_props_test.2018-01-26T09-45-29.xml, xunit.xsd
>
>
> Something like the following should do the job. Instead of
> https://pypi.python.org/pypi/unittest-xml-reporting, it might be possible to
> maybe switch from unittest package to py.test, which has xUnit reports built
> in.
> {noformat}
> diff --git a/src/python/qpid_interop_test/jms_messages_test.py
> b/src/python/qpid_interop_test/jms_messages_test.py
> index 3b54510..3b94f3a 100755
> --- a/src/python/qpid_interop_test/jms_messages_test.py
> +++ b/src/python/qpid_interop_test/jms_messages_test.py
> @@ -24,19 +24,17 @@ Module to test JMS message types across different APIs
> from json import dumps
> from os import getenv, path
> +import xmlrunner
> from proton import symbol
> from qpid_interop_test.test_type_map import TestTypeMap
> @@ -302,7 +300,7 @@ class TestOptions(object):
> Class controlling command-line arguments used to control the test.
> """
> def __init__(self, shim_map):
> - parser = argparse.ArgumentParser(description='Qpid-interop AMQP
> client interoparability test suite '
> + parser = argparse.ArgumentParser(description='Qpid-interop AMQP
> client interoperability test suite '
> 'for JMS message types')
> parser.add_argument('--sender', action='store',
> default='localhost:5672', metavar='IP-ADDR:PORT',
> help='Node to which test suite will send
> messages.')
> @@ -313,6 +311,8 @@ class TestOptions(object):
> parser.add_argument('--broker-type', action='store',
> metavar='BROKER_NAME',
> help='Disable test of broker type (using
> connection properties) by specifying the broker' +
> ' name, or "None".')
> + parser.add_argument('--xunit-report-dir', action='store',
> metavar='REPORTS_DIRECTORY',
> + help='Generate xUnit report into
> REPORTS_DIRECTORY/TEST-<test_name>-<timestamp>.xml')
> type_group = parser.add_mutually_exclusive_group()
> type_group.add_argument('--include-type', action='append',
> metavar='JMS_MESSAGE-TYPE',
> help='Name of JMS message type to include.
> Supported types:\n%s' %
> @@ -421,6 +421,7 @@ if __name__ == '__main__':
> TEST_SUITE.addTest(unittest.makeSuite(test_case_class))
>
> # Finally, run all the dynamically created tests
> - RES = unittest.TextTestRunner(verbosity=2).run(TEST_SUITE)
> +# RES = unittest.TextTestRunner(verbosity=2).run(TEST_SUITE) # type:
> unittest.TextTestResult
> + RES = xmlrunner.XMLTestRunner(verbosity=2).run(TEST_SUITE) # type:
> unittest.TextTestResult
> if not RES.wasSuccessful():
> sys.exit(1)
> {noformat}
> The main "problem" seems that each test has completely separate main method
> and builds up options from scratch. Adding this to every test would mean some
> unnecessary duplication. So maybe refactor this somewhat at first? On the
> other hand, there are good reasons to keep the tests simple...
> What do you think?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
