#34657: Testing assertions `assertContains` and `assertInHTML` should output the
haystack on failure
-------------------------------------+-------------------------------------
Reporter: Thibaud Colas | Owner: nobody
Type: New feature | Status: new
Component: Testing framework | Version:
Severity: Normal | Resolution:
Keywords: testcases, unit | Triage Stage:
tests, HTML, assertions, testing | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Thibaud Colas:
Old description:
> If I use the vanilal Python `assertIn` or `assertRegex`, when the
> assertion fails, it’s very simple to assess what went wrong from the test
> error only:
>
> {{{
> self.assertIn("<b>hey</b>", "<p>Howdy!</p>")
> AssertionError: '<b>hey</b>' not found in '<p>Howdy!</p>'
> }}}
>
> With Django’s `assertContains` and `assertInHTML`, this gets much harder:
>
> {{{
> self.assertInHTML("<b>hey</b>", "<p>Howdy!</p>")
> File "/Users/thibaudcolas/Dev/django/html-testing-with-
> django/.venv/lib/python3.11/site-packages/django/test/testcases.py", line
> 1076, in assertInHTML
> self.assertTrue(
> AssertionError: False is not true : Couldn't find '<b>
> hey
> </b>' in response
> }}}
>
> and:
>
> {{{
> self.assertContains(res, "<b>hey</b>")
> File "/Users/thibaudcolas/Dev/django/html-testing-with-
> django/.venv/lib/python3.11/site-packages/django/test/testcases.py", line
> 660, in assertContains
> self.assertTrue(
> AssertionError: False is not true : Couldn't find '<b>hey</b>' in
> response
> }}}
>
> ---
>
> In both cases, Django doesn’t display the haystack – so I have to waste a
> lot of time going back to my HTML templates to check what they might be
> outputting / or potentially load the same scenario in a browser / or
> manually add `print` statements to my test cases. This is all very time-
> consuming. Instead, it’d be much better if the haystack was just present.
>
> Additionally for `assertInHTML` – it’s annoying that `needle` as
> displayed in the failure message is the parser’s output, which is
> therefore broken up over multiple lines. It’d be much nicer if the
> example above resulted in `AssertionError: False is not true : Couldn't
> find '<b>hey</b>' in response`.
>
> ---
>
> Test suite I used to compare output for reference:
> https://github.com/thibaudcolas/html-testing-with-
> django/blob/39c82c410cce4bea71ac54be27be67057f4d8dd8/testing_tests/tests.py#L5
New description:
If I use the vanilal Python `assertIn` or `assertRegex`, when the
assertion fails, it’s very simple to assess what went wrong from the test
error only:
{{{
self.assertIn("<b>hey</b>", "<p>Howdy!</p>")
AssertionError: '<b>hey</b>' not found in '<p>Howdy!</p>'
}}}
With Django’s `assertContains` and `assertInHTML`, this gets much harder:
{{{
self.assertInHTML("<b>hey</b>", "<p>Howdy!</p>")
File "/Users/thibaudcolas/Dev/django/html-testing-with-
django/.venv/lib/python3.11/site-packages/django/test/testcases.py", line
1076, in assertInHTML
self.assertTrue(
AssertionError: False is not true : Couldn't find '<b>
hey
</b>' in response
}}}
and:
{{{
self.assertContains(res, "<b>hey</b>")
File "/Users/thibaudcolas/Dev/django/html-testing-with-
django/.venv/lib/python3.11/site-packages/django/test/testcases.py", line
660, in assertContains
self.assertTrue(
AssertionError: False is not true : Couldn't find '<b>hey</b>' in response
}}}
---
In both cases, Django doesn’t display the haystack – so I have to waste a
lot of time going back to my HTML templates to check what they might be
outputting / or potentially load the same scenario in a browser / or
manually add `print` statements to my test cases. This is all very time-
consuming. Instead, it’d be much better if the haystack was just present.
Additionally for `assertInHTML` – it’s annoying that `needle` as displayed
in the failure message is the parser’s output, which is therefore broken
up over multiple lines. It’d be much nicer if the example above resulted
in `AssertionError: False is not true : Couldn't find '<b>hey</b>' in
response`.
---
Test suite I used to compare output for reference:
https://github.com/thibaudcolas/html-testing-with-
django/blob/39c82c410cce4bea71ac54be27be67057f4d8dd8/testing_tests/tests.py#L5
And link to Python’s `assertIn` / `assertNotIn` implementations:
https://github.com/python/cpython/blob/101d5ec7d7fe122fa81a377c8ab8b562d1add9ee/Lib/unittest/case.py#L1147-L1159
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34657#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070188c4820664-1c9dbbbc-937a-4cfa-bb5f-14a4dfc7dcb2-000000%40eu-central-1.amazonses.com.
