Hi, What's following here are a few thoughts about things I miss or dislike about pytest, plugins I use, and tox. This is a list of things I personally would like to get fixed during the sprint, but of course that's not the only thing I want to work on, then I wouldn't need a sprint with other people. ;)
Let me mention again this is my *personal* view of things. You might
disagree with some of it. That's okay. Let's not turn this into a
bike-shedding for now, though feel free to reply privately if needed.
This also is not a rant. It's just a reminder for myself and a todo
list. All of this software is awesome! Those are all minor annoyances.
Some of those items have issues open, some of them I haven't told
anyone about yet. A few of them might not be solvable at all.
I thought for some while whether to make this public or not, but
figured, hey, why not.
In no particular order:
pytest
- doesn't have an easy way to do printf-style debugging
(without -s)
- prints the exception before all other report output instead of
after
- segfaults in my testsuite when using pdb (haven't investigated
yet, pdb++ works)
- blows up in weird ways with ImportErrors in plugins
- too much boilerplate with parametrized fixtures
- too much boilerplate with parametrized tests (what about keyword
arguments?)
- has a weirdly structured documentation
- is inconsistent with types and attributes in all of its API
- has an unintuitive warning system, for devs and users
- doesn't work on Windows with colors (for me), prints black on
black
- hides python warnings
pytest-qt
- doesn't have tests yet for its modeltest branch, which is why it
isn't merged yet
- doesn't have a way to check arguments of signals
- doesn't work with PySide2 (which is in development)
pytest-bdd
- doesn't have an easy way to setup/teardown things before/after
all tests in a .feature file
- doesn't have a way of parametrizing tests I can get in my head
- doesn't support markers with arguments in .feature files
- doesn't have gherkin reporting
thinking about writing an alternative which integrates an existing
python bdd framework (behave or lettuce) to profit from everything
those offer
pytest-catchlog
- is still not in pytest-dev or core or renamed to -logging
- still doesn't have pytest-logging functionality integrated
- has no easy way to check a list of log messages
- has no way to fail tests on unexpected error logging messages
pytest-xdist
- breaks my testsuite in ways I don't understand
- doesn't seem to have an easy way for per-job fixtures
pytest-xvfb
- is broken with Qt4
plugin ideas
- good profiling of tests:
What's slow in my testsuite?
What's slow in my application?
- new BDD plugin maybe, see above
tox
- takes too long until my tests start
- doesn't detect changes in requirements.txt files
- doesn't support comments in deps= with newer pip's
- is too verbose
(too much noise with single-line output, needs -q)
- is too quiet (with -v I want to see pip output)
Again - no offense to anyone, and I don't intend to start a
bike-shedding ;)
Looking forward to the sprint!
Florian
--
http://www.the-compiler.org | m...@the-compiler.org (Mail/XMPP)
GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc
I love long mails! | http://email.is-not-s.ms/
signature.asc
Description: PGP signature
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
