Nicholas, Regarding the stack trace filtering, I have been bitten by this too in my custom plugin. You can read the solution here in the mail archives:
https://mail.python.org/pipermail/pytest-dev/2018-March/004399.html If it is not clear enough, just reply and I might be able to help you forward. Ringo On Wed, Apr 18, 2018 at 8:19 PM, Nicholas Williams < nicholas+pyt...@nicholaswilliams.net> wrote: > Hello, all. This is my first post to the mailing list, FWIW. > > I've been a PyTest user for some years, but now I'm working on writing a > rather complex PyTest plugin to facilitate testing our service oriented > architecture at $work. The purpose of the plugin is to collect additional > tests (for which we are using `pytest_collection_modifyitems`) from a > specialized file format that PyTest wouldn't normally understand. I've got > it working beautifully, but I would love to hammer out some finer details > to really make it a clean-running plugin. > > With that in mind, I'm looking for some suggestions / best practices / > advice here on how best to handle these situations: > > Collection Errors > The author of one of these specialized tests, which we call "test plans," > could possibly do a few things wrong that cause more than just making their > test fail. For example, they could get their file syntax incorrect, which > results in a parsing error. At the moment, I catch the parsing error (we're > using PyParsing to parse our test plans) and raise a different exception > about the problem (and I've tried AssertionError), but everything I do > results in a stack trace dumped with "INTERNALERROR>" and the immediate, > abnormal termination of the PyTest process. This contrasts to, for an > analog example, a syntax error in a regular Python test file, which instead > results in a "ERROR collecting" output, while all other unaffected tests > still run and the PyTest process terminates normally. What I'm looking for > here is the "right way" to make my plugin report an "ERROR collecting" > instead of causing an "INTERNALERROR>" when such a problem occurs > collecting these specialized tests. > > Improving the Test Error Report > When tests have errors or failures, PyTest prints out a report of those > errors, and with each one includes an exhaustive stack trace. For our > specialized tests, with some clever manipulation of the test data added to > `items` in `pytest_collection_modifyitems`, I've managed to at least get > it to display a meaningful name for the test that failed so that it's at > least _possible_ for the developer to figure out which test, in which test > file, failed. However, almost the entire stack trace is meaningless and > full of plugin code instead of containing information useful to debugging > the test. Additionally, I would _love_ to be able to add the actual test > file and line number for the failure to the report. Would this be possible? > Or, at least, would it be possible to filter the plugin code out of the > stack trace so that it doesn't distract the developer from simply viewing > the assertion error details? What approach should I take to achieve this? > > Thanks in advance for any help you can provide, > > Nick > > _______________________________________________ > pytest-dev mailing list > pytest-dev@python.org > https://mail.python.org/mailman/listinfo/pytest-dev > > -- *Ringo De Smet* ringo.de.s...@ontoforce.com
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
