On Wed, May 30, 2012 at 7:16 AM, Gary Poster <[email protected]> wrote: > Hi Robert. I asked Brad to go ahead and land his fix for bug 1004625, > because I wanted to get the associated tests back in the parallel > testing bandwagon while we hashed out what to do next. Brad was > reinstating previous behavior, which was and is (at least for now) > desirable, using a small patch, so I approved it.
ok. > This email is to start the process of hashing out next steps. > > You pointed out two bugs: 682772 and 682771. These center around the > fact that you don't want LP to generate duplicate test ids. > > Our thoughts are these, if I understand correctly. I'm counting on > other Yellow Squad folks clarifying my message if I go astray or leave > something out. > > - The Python test runner and test tools have no problem with duplicate > test names. We've encountered this usage pattern on other projects as > well. We'd argue that a test tool that wants to fit in naturally within > the Python testing world would be more convenient if it supported this > existing pattern, of which Launchpad is but one example. There may be a conceptual difference here. Duplicate test names - shrug. Duplicate test ids - much more of a problem. I would say they have a substantial problem when you consider running just one test - a *very* common developer workflow. Launchpad is the only project I've seen with duplicate test ids. Test ids, for clarity, default to the fully qualified class name + . + the test method. This is intrinsically unique unless test multiplication is used, and the only two implementations I've run into are: - the one that doesn't assign unique ids that LP is using for some tests - the one that assigns unique ids via testtools' helper - in testscenarios and bzr. > - testr is the only component in our stack that has trouble with > duplicate test names that we can see. Even so, it almost does just fine > with them. As long as a testrunner expands a single test name to all > associated tests, as ours did and now does again, the tests are run just > fine, all consecutively on a single worker. testr probably does not > record the test time optimally (I'd guess it should aggregate?), and it > does need a bit of work on test counting, but other than that seems to > be fine, at least for roughshod usage. I suspect that a bit of polish > on those two items and a bit of warning documentation would make this a > non-issue for testr users. > > In sum, we are not confident that those two bugs are in fact the right > way to go. We think testr should accommodate the test world, instead. > > Even if you disagree with us, we also don't think that addressing this > one way or another is critical to the project--but as both the client > and the TA, I suspect you might be able to have something to say about > that if you wish. :-) As the duplicates are doctests, and doctests are slow, I suspect this has something to do with the 5 minute idle time you observed in high CPU count parallelism. Just a WAG :). You are right that from a minimal sense 'it will work', but it won't be able to parallelise properly, nor can developers reliably find the variant of a test to run from a backtrace etc. Assigning unique ids is very easy to do using existing heavy lifting code. -Rob -- Mailing list: https://launchpad.net/~yellow Post to : [email protected] Unsubscribe : https://launchpad.net/~yellow More help : https://help.launchpad.net/ListHelp
