Floris, your advice was invaluable. We have created a really fantastic PyTest plugin for our service tests. However, I wanted to share it with you, because it seems to have introduced some type of weird conflict with the PyTest Coverage plugin, and I was hoping maybe you had some insights as to what the problem might be:
Normally, when I run PyTest tests with the PyTest Coverage plugin, everything works as expected in regards to the coverage calculation and the coverage reports. And, with this change to our project, everything STILL works properly IF AND ONLY IF I run tests with `coverage run --source=pysoa setup.py test` followed by `coverage report` (so, without the PyTest Coverage plugin). However, with this change to our project, something VERY weird happens to the coverage report if I rely on the PyTest Coverage plugin. Now, ONLY LINES OF CODE WITHIN FUNCTIONS AND METHODS are marked as covered. The following types of code, which were previously marked as covered (and are clearly covered), are now marked as uncovered (again, only when using the PyTest Coverage plugin): - module-level imports - module-level code that executes on import - module docstrings - module-level class definitions (class ClassName(...):), or class definitions within other class definitions - code that executes within a class on definition - class docstrings - function definition lines (def func_name(...):), but not the code within the functions, which is marked as covered - method definition lines within classes (def method_name(...):), but not the code within the functions, which is marked as covered While the entire pull request is large and daunting, the most relevant part is the PyTest plugin itself, which is contained in a single file "pytest_plugin.py" that you can search for on the page. Here it is: https://github.com/eventbrite/pysoa/pull/87/files Unfortunately, I did not notice this problem with coverage until the bulk of the code was written and tested, so determining what part of the code caused it by removing bits of code is ... impracticable. I'm hoping one of the pytest-cov experts in here will have some idea what the heck is going on and can point me in the right direction. Perhaps I'm doing something wrong, but this smells very strongly of a pytest-cov bug somehow. Thanks, Nick On Fri, Apr 20, 2018 at 2:50 PM, Floris Bruynooghe <f...@devork.be> wrote: > On Thu 19 Apr 2018 at 16:50 -0500, Nicholas Williams wrote: > > > The tests work this way: > > > > - There's a test class that inherits from our specialized test class > > (ServicePlanTestCase), which inherits from `unittest.TestCase` (for > several > > reasons, this detail cannot be altered) > > - That test class can do setup and teardown like a normal test class, but > > it has a static attribute that points to a directory > > - That directory contains one or more files ending in a particular > > extension, and those files each contain one or more tests defined using a > > particular syntax > > I'm not even going to ask how this all came to be this way... but thanks > for giving enough context. > > > Based on the hints you gave me, it sounds like I could do something like > > this: > > https://github.com/pytest-dev/pytest/blob/4678cbeb913385f00cc21b79662459 > a8c9fafa87/_pytest/unittest.py#L14-L22 > > > > Only, instead of checking for inheritance from TestCase, I'd check for > > inheritance from our ServicePlanTestCase, and in that case I would > return a > > new collector object that inherits from _pytest.python.Class, and write > > that new collector class to collect our tests. Am I barking up the right > > tree now? > > pytest_pycollect_makeitem is supposed to create a single item, > while from what you describe you still have two collections nested. You > should probably create a custom collection node for your class, which > then creates a different kind of collection node for each file which in > turn creates the actual test nodes in it's .collect(). >
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
