On Mon, Oct 30, 2023 at 12:51 PM Alassane Yattara < [email protected]> wrote:
> Added pytest init file to configure pytest and generate test reports. > Typo: pytest.ini file? > > Objective > - We propose to integrate the plugin Pytest to generate test reports for > Toaster application. > - To keep configurations to a minimal, we propose to integrate Pytest with > Tox(tox.ini). > - Tox will automate the execution of the Pytest test cases. > > Pros > - Generate test reports > - Create a historical record of test results over time > - Track Toasters stability and quality > - Documentation and Transparency > - Debugging, troubleshooting : identify regressions (input, output,) > > Cons > - Limited predefined report options. Although Pytest provides different > report > formats, its default reports are rather straightforward, We might have to > put extra effort into crafting them ourselves. > One of the benefits of pytest is in fact that report plugins are available and fairly straightforward to implement if we need our own custom report format (although we should avoid custom plugins and use upstream maintained ones wherever possible). We do need to ensure that plain text output is generated for consumption by the AutoBuilder. Fancy html output is going to clutter the testresults.json that will be where these test results will ultimately be stored: https://git.yoctoproject.org/poky/plain/scripts/resulttool https://git.yoctoproject.org/yocto-testresults/ https://docs.yoctoproject.org/test-manual/understand-autobuilder.html?highlight=resulttool#resulttool The reason is that the testresults.json is consumed by tooling to look for regressions. It must be machine readable and easily processed. > Signed-off-by: Alassane Yattara <[email protected]> > --- > lib/toaster/pytest.ini | 19 +++++++++++++++++++ > 1 file changed, 19 insertions(+) > create mode 100644 lib/toaster/pytest.ini > > diff --git a/lib/toaster/pytest.ini b/lib/toaster/pytest.ini > new file mode 100644 > index 00000000..f07076b7 > --- /dev/null > +++ b/lib/toaster/pytest.ini > @@ -0,0 +1,19 @@ > +# -- FILE: pytest.ini (or tox.ini) > +[pytest] > +DJANGO_SETTINGS_MODULE = toastermain.settings_test > + > +python_files = db/test_*.py commands/test_*.py views/test_*.py > browser/test_*.py functional/test_*.py > + > +# --create-db - force re creation of the test database > +# > https://pytest-django.readthedocs.io/en/latest/database.html#create-db-force-re-creation-of-the-test-database > + > +# --html=report.html --self-contained-html > +# https://docs.pytest.org/en/latest/usage.html#creating-html-reports > +# > https://pytest-html.readthedocs.io/en/latest/user_guide.html#creating-a-self-contained-report > +addopts = --create-db --html="Toaster Tests Report.html" > --self-contained-html > + > +# Define environment variables using pytest-env > +# A pytest plugin that enables you to set environment variables in the > pytest.ini file. > +# https://pypi.org/project/pytest-env/ > +env = > + TOASTER_BUILDSERVER=1 > -- > 2.34.1 > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#5921): https://lists.yoctoproject.org/g/toaster/message/5921 Mute This Topic: https://lists.yoctoproject.org/mt/102299113/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/toaster/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
