[Python-ideas] Re: Add @parametrize decorator to unittest library
unittest.subTest, hypothesis @given + pytest From https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests : ``` When there are very small differences among your tests, for instance some parameters, unittest allows you to distinguish them inside the body of a test method using the subTest() context manager. For example, the following test: class NumbersTest(unittest.TestCase): def test_even(self): """ Test that numbers between 0 and 5 are all even. """ for i in range(0, 6): with self.subTest(i=i): self.assertEqual(i % 2, 0) ``` https://hypothesis.readthedocs.io/en/latest/quickstart.html#writing-tests : Think of a normal unit test as being something like the following: 1. Set up some data. 2. Perform some operations on the data. 3. Assert something about the result. Hypothesis lets you write tests which instead look like this: 1. For all data matching some specification. 2. Perform some operations on the data. 3. Assert something about the result. This is often called property-based testing, and was popularised by the Haskell library Quickcheck. It works by generating arbitrary data matching your specification and checking that your guarantee still holds in that case. If it finds an example where it doesn’t, it takes that example and cuts it down to size, simplifying it until it finds a much smaller example that still causes the problem. It then saves that example for later, so that once it has found a problem with your code it will not forget it in the future. Writing tests of this form usually consists of deciding on guarantees that your code should make - properties that should always hold true, regardless of what the world throws at you. Examples of such guarantees might be: https://hypothesis.readthedocs.io/en/latest/details.html#test-statistics : ```bash pytest --hypothesis-show-statistics ``` ``` - during generate phase (0.06 seconds): - Typical runtimes: < 1ms, ~ 47% in data generation - 100 passing examples, 0 failing examples, 0 invalid examples - Stopped because settings.max_examples=100 ``` ```python from hypothesis import given, strategies as st @given(st.integers(), st.integers()) def test_ints_are_commutative(x, y): assert x + y == y + x @given(x=st.integers(), y=st.integers()) def test_ints_cancel(x, y): assert (x + y) - y == x @given(st.lists(st.integers())) def test_reversing_twice_gives_same_list(xs): # This will generate lists of arbitrary length (usually between 0 and # 100 elements) whose elements are integers. ys = list(xs) ys.reverse() ys.reverse() assert xs == ys @given(st.tuples(st.booleans(), st.text())) def test_look_tuples_work_too(t): # A tuple is generated as the one you provided, with the corresponding # types in those positions. assert len(t) == 2 assert isinstance(t[0], bool) assert isinstance(t[1], str) ``` https://hypothesis.readthedocs.io/en/latest/details.html#defining-strategies : > The type of object that is used to explore the examples given to your test function is called a SearchStrategy. These are created using the functions exposed in the hypothesis.strategies module. https://hypothesis.readthedocs.io/en/latest/details.html#defining-strategies : > By default, Hypothesis will handle the global random and numpy.random random number generators for you, and you can register others: https://hypothesis.readthedocs.io/en/latest/supported.html#testing-frameworks : ``` In terms of what’s actually known to work: - Hypothesis integrates as smoothly with pytest and unittest as we can make it, and this is verified as part of the CI. - pytest fixtures work in the usual way for tests that have been decorated with @given - just avoid passing a strategy for each argument that will be supplied by a fixture. However, each fixture will run once for the whole function, not once per example. Decorating a fixture function with @given is meaningless. ``` On Tue, Sep 7, 2021, 09:16 Filipe Laíns wrote: > On Tue, 2021-09-07 at 02:31 +, Leonardo Freua wrote: > > When writing some unit tests with the standard Unittest library, I > missed being > > able to create parameterized tests. This functionality exists in PyTest > > (https://docs.pytest.org/en/6.2.x/parametrize.html) and there is also a > library > > called *parameterized*(https://github.com/wolever/parameterized) which > aims to > > add this functionality. > > > > However, I think it would be beneficial to have a decorator in Python's > standard > > Unittest library. > > > > Note: If this functionality exists natively in Python, please put some > example > > or documentation link below. > > > > Thanks in advance. > > Hi Leonardo, > > Please check out subtests, they allow you to achieve what you are looking > for :) > > https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests > > Cheers, > Filipe La
[Python-ideas] Re: Add @parametrize decorator to unittest library
On Tue, 2021-09-07 at 02:31 +, Leonardo Freua wrote: > When writing some unit tests with the standard Unittest library, I missed > being > able to create parameterized tests. This functionality exists in PyTest > (https://docs.pytest.org/en/6.2.x/parametrize.html) and there is also a > library > called *parameterized*(https://github.com/wolever/parameterized) which aims to > add this functionality. > > However, I think it would be beneficial to have a decorator in Python's > standard > Unittest library. > > Note: If this functionality exists natively in Python, please put some example > or documentation link below. > > Thanks in advance. Hi Leonardo, Please check out subtests, they allow you to achieve what you are looking for :) https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AJSJKC4PD4KWS53IJLTWORHC7QZGJXDG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add @parametrize decorator to unittest library
07.09.21 05:31, Leonardo Freua пише:
> When writing some unit tests with the standard Unittest library, I missed
> being able to create parameterized tests. This functionality exists in PyTest
> (https://docs.pytest.org/en/6.2.x/parametrize.html) and there is also a
> library called *parameterized*(https://github.com/wolever/parameterized)
> which aims to add this functionality.
>
> However, I think it would be beneficial to have a decorator in Python's
> standard Unittest library.
>
> Note: If this functionality exists natively in Python, please put some
> example or documentation link below.
It was discussed before and subTest() was added as more general alternative.
Instead of
@parametrize("a", [1, 2, 10])
def test(self, a):
...
you should write
def test(self):
for a in [1, 2, 10]:
with self.subTest(a=a):
...
The advantage is that you can generate parameters at run time. You can
add a code before and after the loop. Use several sequential loops.
There are also other use cases for subTest().
The disadvantage is that it adds at least two indentation levels (more
if you use nested loops) to the test code. A simple decorator would be
more convenient in simple cases.
It is easy to implement the decorator in terms of subtests, but
currently there are some issues with outputting results of subtests:
https://bugs.python.org/issue25894
https://bugs.python.org/issue30856
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/5ES2KP34Z4PC75MBAHAAL4JYRQJ7UDJ6/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add @parametrize decorator to unittest library
On Mon, Sep 6, 2021 at 21:45 Christopher Barker wrote: > Just use pytest. > For third party code I agree, it’s the way to go. —Guido -- --Guido (mobile) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/75MAGYGI3NKUTVFWQ4P3QHO5OX6JQORF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add @parametrize decorator to unittest library
I believe he is looking for something like pytest’s parameterize: https://docs.pytest.org/en/6.2.x/example/parametrize.html This is actually pretty basic functionality for writing DRY tests, a key missing feature. Frankly though, unittest is painfully unpythonic and hard to extend. I’ve given up. Just use pytest. -CHB On Mon, Sep 6, 2021 at 7:56 PM Steven D'Aprano wrote: > Hi Leonardo, > > On Tue, Sep 07, 2021 at 02:31:26AM -, Leonardo Freua wrote: > > > When writing some unit tests with the standard Unittest library, I > > missed being able to create parameterized tests. > > Could you please explain what you mean by "parameterized tests", and how > you would use a decorator for it? What you mean by it may not be what > other people understand it to be. > > > > -- > Steve > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/CA7ZUSS7OXSM42IWZUXRKYNKVQG6MF2F/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/54DSL6XLGAYMQ3DYVCE7HMQGG3HSK5NU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add @parametrize decorator to unittest library
Hi Leonardo, On Tue, Sep 07, 2021 at 02:31:26AM -, Leonardo Freua wrote: > When writing some unit tests with the standard Unittest library, I > missed being able to create parameterized tests. Could you please explain what you mean by "parameterized tests", and how you would use a decorator for it? What you mean by it may not be what other people understand it to be. -- Steve ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CA7ZUSS7OXSM42IWZUXRKYNKVQG6MF2F/ Code of Conduct: http://python.org/psf/codeofconduct/
