Hi Florian Longer ago you had some good points about selecting only tests affected by recent changes as a basic functionality for our plugin - testmon.
We did more testing and thinking about these difficult questions and I think they will not be that bad. > I also thought about using something like coverage to only run affected > tests. How much testing did you do in your prototype? E.g. pytest test suite itself runs pretty nicely with testmon. > Do you keep track of (pip/setuptools) installed packages (versions)? Not yet, but I'm pretty sure a simple detection of changes under sys.path that would trigger a full test run should be easy to implement and quick enough. Did you try changes at various levels like new files, modules, classes, > metaclasses, mocks etc? Python is very dynamic as you most likely know :) > * files: sure * modules, classes, metaclasses, mocks: Unless I'm missing something this is not necessary. All of the dynamic constructs are created deterministically in projects py files. If none of the files involved in execution of a test is changed, then all of the dynamic constructs will be the same in the next test run. Of course I might be missing somethings, so if somebody has a good case please, let us know. These things cause trouble for coveragepy (multiprocessing and gevent stuff is beeing solved): http://nedbatchelder.com/code/coverage/trouble.html These are my thoughts about the dependencies of a test: https://github.com/tarpas/testmon#thoughts Thanks for help, Tibor testmon - make your tests a breeze to execute http://igg.me/at/testmon > > Regards, > Florian Schulze >
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
