New issue 449: better indicate that there were xpassed test results in a test
run
https://bitbucket.org/hpk42/pytest/issue/449/better-indicate-that-there-were-xpassed
Jurko Gospodnetić:
This is a usability related enhancement suggestion.
When running tests what I really want to know first is:
* Did anything unexpected happen?
which includes both:
* Whether there were any test failures?
* Whether any tests marked ```xfail``` suddenly started passing?
The first I can get easily at first glance - either by running the tests with
the ```-x``` option or by checking the final summary line color (green/red).
The latter is a problem though because in order to find whether there were any
```xpassed``` test I have to concentrate a lot harder and either:
* Scan the test summary for ```xpassed``` test result indicators (capital
letter ```X```) which can be difficult to discern from expected ```xfailed```
(lower letter ```x```) results.
* Read the final colored summary line to see if any ```xpassed``` test results
occurred.
* Run the tests with the '-r X' option and read whether the summary displays
any ```xpassed``` test results.
One of my projects has a lot of tests marked ```xfail``` and it started to bug
me that I often waste a lot of time and interrupt my flow & concentration by
having to check the test results in detail just to see if there were any
```xpassed``` test results.
My suggestion would be to:
* Use a different color (e.g. blue?) if you would otherwise color it green by
at least one ```xpassed``` test result was encountered.
* Use an exit code other that a simple 0=success in such cases.
* Possibly color individual ```failed``` (```F```) & ```xpassed``` (```X```)
test result indicators red or red/blue.
You might not want to use the new ```xpassed``` result related coloring when
running with disabled assertions (```-O```) and you displayed a warning about
this possibly causing failing tests to be marked as passed, e.g. when running
using an older Python interpreter version.
The whole enhancement could possibly be made configurable as well.
Hope this helps.
Best regards,
Jurko Gospodnetić
_______________________________________________
pytest-commit mailing list
pytest-commit@python.org
https://mail.python.org/mailman/listinfo/pytest-commit
