Firstly, I'd like to thank Holger Krekel and the rest of the pytest team
for all their hard work (including the very good documentation and videos).
I'm a novice pytest user and I wanted to run a scenario by more
experienced pytest users, before potentially going down the wrong path.
I would appreciate feedback on the helper functions below (limited
testing suggests that they do work) and suggestions for alternative
approaches.
Thank you,
Peter Santoro
============================
GIVEN:
Pytest will be used to automate the running and evaluation of tests for
a system that is not written in Python. I have already prototyped this
in Python without using pytest. I'm currently converting my prototype
code to conform with pytest APIs, rules, and conventions. So far so good.
There will eventually be hundreds of tests, each of which requires the
copying of a test specific directory tree structure from a source
location to a destination location (required by the system being
tested). I want to avoid copying any extra directory
trees, because:
1) it will increase the time it takes the copying fixture(s) to run
2) it will also cause the system actually running the tests
(outside of pytest, but run via a fixture) to do unnecessary work (i.e.
waste time by running too many tests)
I also want to limit the amount of work required to create each test
function, so I plan to use helper functions that utilize the unique
docid to collect the specific test results from the system being
tested and use asserts to compare the result with the previously stored
expected results.
The proposed test function naming convention is
test_doctype_docyear_schema_docid, where:
doctype = document type
docyear = document's year
schema = document's xml schema version
docid = globally unique document id
In addition to using pytest markers, this naming convention (via pytest
-k command line option), will allow for easy selection of the tests to
be run from the pytest command line.
The proposed source location directory tree structure (holds test input
files and expected results record):
docyear_schema
|
------- doctype
|
------------ docid
The destination directory tree structure is fixed per the software that
will be actually running the tests:
location based on doctype
|
------------ docid
POSSIBLE SOLUTIONS TO DIRECTORY TREE COPYING FIXTURE(S):
1) Write a separate copying fixture function for each test;
however, this seems like too much work.
2) Use pytest markers to logically group tests (see marker related
helper functions below), but this becomes problematic if only a small
subset of the marked group(s) needs to be run.
3) Use a suitable test function naming convention that would allow
a single copying fixture function to handle the copying for all selected
tests. This of course requires that the copying fixture knows what
tests actually requested itself (see fixture_test* functions below).
This is my preferred approach.
# marker helper functions
def markers(node):
for v in node.keywords.values():
if isinstance(v, (MarkDecorator, MarkInfo)):
yield v
def marker_names(node):
return [m.name for m in markers(node)]
def request_markers(request):
for n in request.node.items:
yield from markers(n)
def request_marker_names(request):
return [m.name for m in request_markers(request)]
# helper functions to get the applicable tests for a given fixture
def fixture_test_names(name, request):
return [n.name for n in fixture_tests(name, request)]
def fixture_tests(name, request):
for n in request.node.items:
if name in n.fixturenames:
yield n
# sample hook function using a helper function to get list of marker
names to make decisions on
def pytest_collection_modifyitems(session, config, items): # hook
for n, item in enumerate(items):
print(n, item.nodeid, list(marker_names(item)))
if 'func2' in item.nodeid:
del items[n]
# sample fixture using helper functions to get tests
associated with itself
@pytest.fixture(scope='session')
def fcopy(request):
"""copy files for selected tests that use the fcopy fixture"""
print(list(fixture_tests('fcopy', request)))
print(fixture_test_names('fcopy', request))
# sample fixture using helper functions to get list of marker names
@pytest.fixture(scope='session')
def fcopy2(request):
"""copy files associated with certain marker(s)"""
print(list(request_markers(request)))
print(request_marker_names(request))
# sample test functions that use the fcopy fixture
@pytest.mark.mymarker1
@pytest.mark.mymarker2
def test_function1(fcopy):
pass
@pytest.mark.mymarker3
@pytest.mark.mymarker4
def test_function2(fcopy):
pass
_______________________________________________
Pytest-dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/pytest-dev
