On Mon, Jun 17, 2013 at 6:02 PM, Henry Gessau <[email protected]> wrote:
> On Mon, Jun 17, at 7:01 pm, Joe Gordon <[email protected]> wrote: > > Hi All, > > > > A patch to move the run_tests.sh script into oslo-incubator > > (https://review.openstack.org/#/c/32736/), has brought up the bigger > > question of what is the future of './run_tests.sh.' > > > > This seems like a topic that directly affects all developers, so it > > sounded like it should be brought to the Mailing list. > > > > Reasons to keep run_tests.sh: > > > > * there is no possibility to run tests with testtools instead of testr. > > This feature allows us to use the debugger. > > * in some projects tox doesn't use a testr wrapper to report progress > > while tests are running > > * building the sdist can be slow (slow == noticeable) > > > > > > Monty's Rant: > > > > "So, I'm SUPER torn on this. > > > > On the one hand, we use tox in the gate, and run_tests.sh hides that from > > people and then they get confused about why some change they made to > > run_tests.sh doesn't get reflected in the reality of the project. > > > > On the other hand, what Victor says is true. We need to land a couple of > > changes to upstream tox that would allow it to not run develop each time > - > > but we seem to have a hard time doing that. > > > > On the third hand, we'd doing an INSANE amount of work to make working > > with testing in a venv "easy" and what we've wound up with is a situation > > where we have multiple competing incompatible ways of doing things. > > > > Here's what I do: source .tox/py27/bin/activate testr run --parallel > testr > > run --parallel testr run some-test testr run --failing deactivate > > > > Sometimes, if I'm being fancy, I'll do: virtualenv foo foo/bin/pip > install > > -r requirements.txt -r test-requirements.txt > > > > I believe vish just installs the depends directly on his box and skips > > venvs altogether. > > > > It turns out that testr is a powerful tool with a nice UI. Instead of > > putting wrappers around it, perhaps we should add support for our output > > stream thing we like to it upstream. We should also add support to it for > > dropping into a debugger directly, so that the testtools issue goes away. > > Then we should land a patch to upstream tox to get it to do setup.py > > develop instead of sdist+ install. > > > > If we did that, we could have "run tox" be the simple answer for anyone > > who just wants to run whatever the gate runs, because that will always > > work. And if they want to do fancier things, they should learn about > venvs > > and testr. > > > > However, if we ARE going to keep a run_tests.sh file in our trees, we > > should certainly have a single copy and have it behave consistently. > > > > Please note that some projects actually do just have run_tests.sh as a > > thin wrapper around tox." > > > > > > What do you think? Should we keep run_tests as is and port to oslo, or > > should we revisit the role of run_tests.sh? > > > > > My vote is for run_tests.sh to contain just one line, "cat TESTING", and > ensure the instructions are detailed and up to date. If course, it would be > nice if all the projects gravitated towards the same TESTING. > > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > Post conversation with Lifeless and Clarkb on IRC, I'll change my answer slightly: As long as I can do the following: 1. Run individual tests easily 2. Drop into PDB by setting a break-point 3. Run WITHOUT virtual env 4. Not have to download a bunch of stuff or run some monster init every time I run tests on a fresh clone of a repo (see item 3) I don't care if that's done via a wrapper script that calls a framework that supports the option (well I do care but we're already there so I'd prefer not to take it away), better documentation of how to run tests and how the frameworks operate or whatever.
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
