Hi Aaron, On 12-09-21 02:38 PM, Aaron Bentley wrote: > > That duration is similar to bzr's test suite. With bzr, we've always > been able to run in the traditional PQM mode where PQM runs the tests, > and if they fail, the code doesn't land. > > I bet that would work with Launchpad too, which would allow us to > simplify a lot of our infrastructure-- launchpad/devel and > launchpad/stable become a single branch, we don't need buildbot > anymore, and we don't need to use "ec2 land". We can just lp-land > directly. > > When test runs were longer, this mode caused unfortunate queueing, but > with 35-minute test runs, we can do 41 test runs a day, and the most > landings I see this month is 11 (on Sept 12). Most days are more like 6. >
I agree that going back to pre-commit merge is one thing we could try. There is a caveat in your data though. You are counting the number of test runs processed by buildbot in a day. The number that is truly important is the number of ec2test submission daily. Because only once a successful ec2 test run has happened does it gets send to buildbot. What you are proposing (and what was happening before we switched to buildbot) is that developers simply use the landing architecture as a convenient test runner. One problem in the old days was that a lot of queued landings would fail simply because the tests hadn't been run, not because of failing tests because of a integration error or intermittent failure. (Although we had also some of those.) I don't know what's the effort required to set a tarmac instance that can run parallelized tests. (Unfortunately, it probably requires scarce webops resource also). But I'd be willing to try an experiment around that if it's cheap. To achieve the similar flow you want, we can also make ec2test run tests in parallel in EC2. On the big instances with 32 cores, Yellow was seeing a ~50 mins test run in EC2. That would put us well into range of writing and deploying code in the same day. (And deploying this requires 0 webops involvement). (And one doesn't exclude the other.) -- Francis J. Lacoste francis.laco...@contre.com
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
