jenkins-bot has submitted this change. (
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1315873?usp=email )
Change subject: docs: improve tests package documentation
......................................................................
docs: improve tests package documentation
Change-Id: I2101eaa3db8686bad54eaefd52a67d1dd696c555
---
M docs/tests_ref/aspects.rst
M docs/tests_ref/basepage.rst
M docs/tests_ref/index.rst
M docs/tests_ref/precommit.rst
M docs/tests_ref/pytest.rst
A docs/tests_ref/tests.rst
M docs/tests_ref/utils.rst
M tests/README.rst
M tests/__init__.py
9 files changed, 211 insertions(+), 49 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/docs/tests_ref/aspects.rst b/docs/tests_ref/aspects.rst
index 948faf6..7acdf29 100644
--- a/docs/tests_ref/aspects.rst
+++ b/docs/tests_ref/aspects.rst
@@ -1,8 +1,9 @@
-********************
-tests.aspects module
-********************
+***********************************************
+:mod:`tests.aspects` --- Test behaviour aspects
+***********************************************
.. automodule:: tests.aspects
+ :synopsis: Reusable test behaviour aspects for the Pywikibot test suite.
:members:
:undoc-members:
:show-inheritance:
diff --git a/docs/tests_ref/basepage.rst b/docs/tests_ref/basepage.rst
index 97e2d09..463ab40 100644
--- a/docs/tests_ref/basepage.rst
+++ b/docs/tests_ref/basepage.rst
@@ -1,8 +1,9 @@
-*********************
-tests.basepage module
-*********************
+***********************************************
+:mod:`tests.basepage` --- BasePage test helpers
+***********************************************
.. automodule:: tests.basepage
+ :synopsis: BasePage tests subclasses
:members:
:undoc-members:
:show-inheritance:
diff --git a/docs/tests_ref/index.rst b/docs/tests_ref/index.rst
index 2d4eefc..750b95f 100644
--- a/docs/tests_ref/index.rst
+++ b/docs/tests_ref/index.rst
@@ -12,6 +12,7 @@
.. toctree::
:titlesonly:
+ tests
aspects
basepage
utils
diff --git a/docs/tests_ref/precommit.rst b/docs/tests_ref/precommit.rst
index 1fb5769..8c1e820 100644
--- a/docs/tests_ref/precommit.rst
+++ b/docs/tests_ref/precommit.rst
@@ -1,8 +1,9 @@
-**********************
-precommit hooks module
-**********************
+***************************************
+:mod:`tests.hooks` --- Pre-commit hooks
+***************************************
.. automodule:: tests.hooks.copyright_fixer
+ :synopsis: Check and update copyright headers in Pywikibot source files
:members:
:undoc-members:
:show-inheritance:
diff --git a/docs/tests_ref/pytest.rst b/docs/tests_ref/pytest.rst
index 5d062af..b965fc9 100644
--- a/docs/tests_ref/pytest.rst
+++ b/docs/tests_ref/pytest.rst
@@ -1,8 +1,9 @@
-********************
-pytest plugin module
-********************
+****************************************
+:mod:`conftest` --- Pytest plugin module
+****************************************
.. automodule:: conftest
+ :synopsis: Pytest fixtures and hooks integrating the Pywikibot test suite
with pytest
:members:
:undoc-members:
:show-inheritance:
diff --git a/docs/tests_ref/tests.rst b/docs/tests_ref/tests.rst
new file mode 100644
index 0000000..7057dd3
--- /dev/null
+++ b/docs/tests_ref/tests.rst
@@ -0,0 +1,9 @@
+*****************************************
+:mod:`tests` --- Core test infrastructure
+*****************************************
+
+.. automodule:: tests
+ :synopsis: Core infrastructure for running and organizing the Pywikibot
test suite
+ :members:
+ :undoc-members:
+ :show-inheritance:
diff --git a/docs/tests_ref/utils.rst b/docs/tests_ref/utils.rst
index 72b7900..55c0e23 100644
--- a/docs/tests_ref/utils.rst
+++ b/docs/tests_ref/utils.rst
@@ -1,8 +1,9 @@
-******************
-tests.utils module
-******************
+*************************************
+:mod:`tests.utils` --- Test utilities
+*************************************
.. automodule:: tests.utils
+ :synopsis: Test utility funktions
:members:
:undoc-members:
:show-inheritance:
diff --git a/tests/README.rst b/tests/README.rst
index d5bd42a..905c1ae 100644
--- a/tests/README.rst
+++ b/tests/README.rst
@@ -5,8 +5,8 @@
The Pywikibot tests are based on the `unittest framework
<https://docs.python.org/3/library/unittest.html>`_.
-The tests package provides a function load_tests that supports the
-`load tests protocol
+The tests package provides a function :func:`tests.load_tests` that supports
+the `load tests protocol
<https://docs.python.org/3/library/unittest.html#load-tests-protocol>`_.
The default ordering begins with tests of underlying components, then tests
site and page semantics, and finishes with tests of the scripts and finally
@@ -30,7 +30,7 @@
**pytest**
-.. note:: Python 3.10 or higher is required to run pytest.
+.. note:: Running the test suite with pytest requires Python 3.10 or higher.
::
@@ -110,17 +110,17 @@
PYWIKIBOT_TEST_NO_RC=1
**PYWIKIBOT_TEST_OAUTH**
- This environment variable holds the Oauth token. It is set by
+ This environment variable holds the OAuth token. It is set by
``oauth_tests-ci.yml`` CI config file and is solely used by
:source:`tests/oauth_tests`. You can use it for your private tests. The
environment variable must contain consumer key and secret and access
key and secret delimited by ``:`` as::
- PYWIKIBOT_TEST_OAUTH=consumer_key:consumer_secret:access_key:access:secret
+ PYWIKIBOT_TEST_OAUTH=consumer_key:consumer_secret:access_key:access_secret
**PYWIKIBOT_TEST_QUIET**
- This environment variable can be set for quit mode. It prevents output by
- test package, i.e. 'max_retries reduced from x to y'. It is used be the
+ This environment variable can be set for quiet mode. It prevents output by
+ test package, i.e. 'max_retries reduced from x to y'. It is used by the
:func:`tests.utils.execute` test runner. To enable it for other tests use::
PYWIKIBOT_TEST_QUIET=1
@@ -230,33 +230,44 @@
def test_patch(self):
self.assertEqual('pong', http_ping())
+
Contributing tests
==================
-Test modules should be named according to the pywikibot that is being tested.
-e.g. the module pywikibot.page is tested by tests.page_tests.
+The test package distinguishes between framework tests, which cover Pywikibot
+framework components, and script tests, which cover individual Pywikibot
+scripts.
-New test classes should be added to the existing test modules unless it
-tests a new component of pywikibot.
+Test modules should be named according to the Pywikibot component being
+tested. For example, the module :mod:`pywikibot.page` is tested by
+:source:`tests/page_tests`.
-All test classes must be a subclass of tests.aspects.TestCase, which uses a
-metaclass to dynamically check the test can be run on a specified site, or
-run a test on multiple sites.
+New test classes should be added to the existing test modules unless they
+test a new component of Pywikibot.
+
+All test classes must be a subclass of :class:`TestCase
+<tests.aspects.TestCase>`. Its metaclass validates the declared test
environment
+and dynamically creates the required site objects. Tests can declare a specific
+site, multiple sites, or other behaviour attributes as described in
+:ref:`Test behaviour attributes`.
+
Test sites
----------
-If a test depends on a specific site, add class attributes 'family' and code'.
+If a test depends on a specific site, add the class attributes ``family``
+and ``code``:
::
family = 'wikipedia'
code = 'en'
-Once declared, the Site object can be accessed at self.site.
+Once declared, the Site object is available as ``self.site``.
-If a test requires multiple specific sites, add a class attribute 'sites'.
+If a test requires multiple specific sites, define the ``sites`` class
+attribute. Each key becomes a separate test variant.
::
@@ -271,30 +282,139 @@
}
}
-To obtain the Site object, call self.get_site with the key given to the site.
+To obtain the Site object, call :meth:`get_site<tests.TestCase.get_site>`
+with the key given to the site:
::
self.get_site('itwikt')
-For tests which require network access to a website which is not an APISite,
-the class attribute 'sites' may include a hostname.
+If a test method accepts the site key as its second positional argument,
+the metaclass creates one test for each entry in ``sites`` and passes the
+corresponding key to the test method:
+
+::
+
+ def test_something(self, site_key):
+ site = self.get_site(site_key)
+
+For tests which require network access to a host which is not a MediaWiki
+API site, the class attribute 'sites' may include a hostname:
::
sites = {
- 'wdq':
+ 'wdq': {
'hostname': 'query.wikidata.org',
}
}
+If no API site is used and only hosts are require, you may define
+``hostname`` or ``hostnames``:
-Other class attributes
-----------------------
+::
-- ``net = False``: test class does not use a site
-- ``dry = True``: test class can use a fake site object
-- ``cached = True``: test class may aggressively cache API responses
-- ``login = True``: test class needs to login to site
-- ``rights = '<rights>'``: test class needs specific rights. Multiple rights
must be delimited with ``,``.
-- ``write = True``: test class needs to write to a site
+ hostname = 'query.wikidata.org'
+
+or:
+
+::
+
+ hostnames = [
+ 'query.wikidata.org',
+ 'example.org',
+ ]
+
+The hosts are added to the test site's definitions and checked for network
+availability.
+
+
+Test behaviour attributes
+-------------------------
+
+The following class attributes control the behaviour of a test class.
+Attributes which enable additional functionality are normally set to
+``True``. Attributes which are not set keep their default behaviour.
+
+Some attributes implicitly enable additional behaviour or add mixins to
+the test class.
+
+
+Caching
+~~~~~~~
+
+``cached = True``
+ The test class may aggressively cache API responses. This adds
+ :class:`ForceCacheMixin<tests.aspects.ForceCacheMixin>`. ``cached``
+ is intended for read-only tests and must not be combined with
+ ``write = True``.
+
+
+Network access
+~~~~~~~~~~~~~~
+
+``net = True``
+ The test class explicitly requires network access.
+
+``net = False``
+ The test class explicitly declares that no network access is used.
+
+ Test classes which do not use a site must explicitly define ``net``.
+
+``site = False``
+ The test class does not use a Site object. This adds
+ :class:`DisableSiteMixin<tests.aspects.DisableSiteMixin>` and prevents
+ calls to :func:`pywikibot.Site`. ``site = False`` is commonly combined
+ with ``net = False`` for tests which do not access a site or the network.
+
+Disconnected site tests
+~~~~~~~~~~~~~~~~~~~~~~~
+
+``dry = True``
+ The test class uses disconnected Site objects instead of accessing real
+ sites. This adds :class:`DisconnectedSiteMixin
+ <tests.aspects.DisconnectedSiteMixin>`. ``dry`` implicitly disables
+ network access (equivalent to ``net = False``).
+
+
+Authentication
+~~~~~~~~~~~~~~
+
+``login = True``
+ The test class requires authentication on the configured site. This
+ adds :class:`RequireLoginMixin<tests.aspects.RequireLoginMixin>`.
+
+``oauth = True``
+ The test class uses OAuth authentication when authentication is required.
+
+``rights = '<rights>'``
+ The test class requires specific user rights. Multiple rights must be
+ separated by commas. Setting ``rights`` implicitly enables ``login = True``
+ and adds :class:`NeedRightsMixin<tests.aspects.NeedRightsMixin>`.
+
+
+Writing tests
+~~~~~~~~~~~~~
+
+``write = True``
+ The test class performs write operations on a site. This adds
+ :class:`SiteWriteMixin<tests.aspects.SiteWriteMixin>`. Setting ``write``
+ implicitly enables ``login = True``. Write tests require explicit enabling
+ through the test environment ``PYWIKIBOT_TEST_WRITE``.
+
+
+Script execution
+~~~~~~~~~~~~~~~~
+
+``pwb = True``
+ The test class invokes scripts through :mod:`pwb`. Test classes using
+ ``pwb`` normally require a configured site. If a ``pwb`` test does not
+ use a site, it must explicitly define ``site = False``.
+
+
+Wikibase tests
+~~~~~~~~~~~~~~
+
+``wikibase = True``
+ The test class requires sites with a Wikibase data repository. This
+ is used by :class:`WikibaseTestCase<tests.aspects.WikibaseTestCase>`.
diff --git a/tests/__init__.py b/tests/__init__.py
index 851a402..c84212e 100644
--- a/tests/__init__.py
+++ b/tests/__init__.py
@@ -3,15 +3,38 @@
#
# Distributed under the terms of the MIT license.
#
-"""Package tests."""
+"""Core infrastructure for the Pywikibot test suite.
+
+This module defines the main groups of test modules and implements the
+`unittest load tests protocol
+<https://docs.python.org/3/library/unittest.html#load-tests-protocol>`_.
+It controls the order in which test modules are loaded and provides the
+entry point used by the test runner.
+
+It also provides helpers for locating test data paths and for controlling
+API request caching during tests.
+
+The test modules are grouped into library tests, which cover Pywikibot
+library components, and script tests, which cover Pywikibot scripts. The
+corresponding module collections are available as
+:data:`library_test_modules` and :data:`script_test_modules`.
+
+Shared test infrastructure such as test case classes and reusable test
+components is implemented in :mod:`tests.aspects`, :mod:`tests.basepage`,
+and :mod:`tests.utils`.
+
+For information about running tests, writing new tests, and available test
+options, see :ref:`Pywikibot tests`.
+"""
from __future__ import annotations
__all__ = (
- 'create_path_func', 'join_cache_path', 'join_data_path',
- 'join_html_data_path', 'join_images_path', 'join_pages_path',
- 'join_root_path', 'join_xml_data_path', 'patch_request', 'unittest_print',
- 'unpatch_request',
+ 'TestRequest', 'create_path_func', 'collector', 'join_cache_path',
+ 'join_data_path', 'join_html_data_path', 'join_images_path',
+ 'join_pages_path', 'join_root_path', 'join_xml_data_path',
+ 'library_test_modules', 'load_tests', 'patch_request',
+ 'script_test_modules', 'unittest_print', 'unpatch_request',
)
import functools
@@ -65,6 +88,8 @@
# Find the root directory of the checkout
_pwb_py = join_root_path('pwb.py')
+#: Names of test modules covering Pywikibot framework components.
+#: The corresponding test files end with ``_tests.py``.
library_test_modules = {
'api',
'basesite',
@@ -143,6 +168,8 @@
'xmlreader'
}
+#: Names of test modules covering Pywikibot scripts.
+#: The corresponding test files end with ``_tests.py``.
script_test_modules = {
'add_text',
'archivebot',
--
To view, visit
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1315873?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gerrit.wikimedia.org/r/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I2101eaa3db8686bad54eaefd52a67d1dd696c555
Gerrit-Change-Number: 1315873
Gerrit-PatchSet: 4
Gerrit-Owner: Xqt <[email protected]>
Gerrit-Reviewer: Xqt <[email protected]>
Gerrit-Reviewer: jenkins-bot
_______________________________________________
Pywikibot-commits mailing list -- [email protected]
To unsubscribe send an email to [email protected]
