On Wed, Jul 1, 2009 at 2:18 AM, Stefan Weigel<[email protected]> wrote: > No doubt, testing is necessairy. But one cannot produce quality by testing. > High Quality is a result of *preventing* bugs.
But thorough automated testing can help by a) pointing out regressions early so they can be fixed before they infect the trunk, and b) rubbing the noses of those who committed the regressions in their mess closer to the time they did it; that way they get the message better that they might need to improve their quality processes. For instance, yesterday a new programmer on Chromium who had just gotten his change to pass our automated voluntary presubmit smoke test (yay) made a tiny change and then checked in without testing (boo). The trunk's massive automated testing caught it quickly and dramatically, and the IRC channel filled up with chatter about the programmer. He heard about this, came onto IRC, apologized profusely, and promised to never make that mistake again. All this within 20 minutes of him making the mistake. How quickly do OOo programmers get feedback about easy regressions checked into CWSs? Nobody likes to be told how to run their projects, but as long as we're on the subject, here are another couple bits of process from Chromium that might be worth thinking about for OOo: Chromium doesn't have feature branches (= CWS's, I think). Features and bugfixes land immediately on the trunk. This subjects them to massive automated testing. If even a single test fails, we close the tree to new checkins until the change is reverted or fixed, the test is suppressed due to flakiness, or a bug is filed and the test marked as a known regression and tracked in our bug tracking system. This cuts down on regressions due to merge problems. While OOo's build process has surely improved over the years, it might be worth looking at further simplifications. It is trivial for new Chromium developers to download and build the source on all three platforms. On Windows, all you need to do is press F5 in Visual Studio, and out pops a working build of Chromium. On Linux, all you need to do is type 'make -j4 chrome'. And it's that easy in Xcode on the Mac, too. You don't need to install Cygwin on Windows; the proper subset of that (properly modified so as not to conflict with the system cygwin) is already included in the checked out source tree. This increases the pool of available developers. Chromium recently switched (mostly) to a cmake-like tool called gyp so that the native build system for all platforms is generated automatically from a single source, and is updated whenever you sync new sources. This is working out very nicely. (I don't recall why we couldn't use cmake instead of inventing our own -- it could be as simple as the guy doing our build tools loves python -- but at least gyp is open source; http://gyp.googlecode.com.) This makes it easier to program on one platform without breaking the others. Chromium doesn't use a private bug tracking system (except for security bugs) or a private source code control system. All development is done totally in the open. When I was helping with OOo QA, I got the distinct impression that lots was being done in a walled garden that I had no access to. As a fulltime Chromium developer, I might have a skewed view of things, but at least we're trying to do everything out in the open. (To the extent of even using unforked webkit and doing our webkit changes on the webkit trunk and participating fully in that project.) This increases scrutiny of and finds bugs early in our changes, and makes it easier for us to stay current with fixes from upstream. I don't know how much of that is applicable to or an improvement on OOo's process, but it seems to be working well for us. - Dan --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
