On Sat, Apr 3, 2010 at 11:30 PM, Darin Adler <da...@apple.com> wrote: > On Apr 3, 2010, at 10:36 AM, Adam Barth wrote: >> Keeping the tree green will require a cultural shift in the project, but I >> think the near term costs of changing the culture are outweighed by the long >> term gains in productivity. > > Typically a cultural shift would come after some sort of group discussion, > rather than being announced on the mailing list as a fait accompli.
We've been discussing this on the mailing list for a while. Here are some recent threads: * https://lists.webkit.org/pipermail/webkit-dev/2009-September/010023.html In which the conclusion is that we should fix test flakiness rather than skipping the tests or letting the flakiness persist. * https://lists.webkit.org/pipermail/webkit-dev/2010-January/011430.html In which we manually go through the process outlined above using webkit-dev instead of more targeted channels. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011561.html In which we discuss how best to draw attention to buildbot failures. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011562.html This email in particular, in which Maciej asks someone to build a bot that does precisely what sheriffbot does: [[ I'd actually like to see it email a mailing list, in addition to the individuals it guesses are to blame. That could be either webkit-dev or a new list. Maybe some won't want the spam but I bet a lot of people would like to find out about every build break. If it's at all possible, it would be great to email all of the patch author, the reviewer and the committer (if different from the patch author). I also think it would be neat if we could have a bot that alerts about build breaks on IRC in #webkit. And finally, it might be good to have extra notice if a build remains broken for some time (every 24 hours maybe?) ]] * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011749.html This long thread where we discuss the motivations for keeping the tree green and the possibility of using rollouts more than we already are. Here are some important messages from that thread: * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011765.html In which Dimitri Glazkov supports a "rollout first" strategy. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011767.html In which Maciej says that is polite (but not mandatory) to ask the responsible person first. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011768.html In which Nikolas Zimmermann expresses concern about automatic rollouts and asks for a 30 minute window to fix it. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011771.html In which Eric Seidel outlines a sheriffbot algorithm that's similar (but slightly more agressive) than what we've built. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011774.html In which Jeremy Orlow supports that algorithm. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011776.html In which Alexey Proskuryakov asks for #webkit notification in addition (which we did). * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011790.html In which Maciej supports Geoffrey Garen in requesting that we build a better failure notification system before we build an automatic rollout system. * https://lists.webkit.org/pipermail/webkit-dev/2010-February/011792.html In which Maciej again asks that we have a sheriff (although I think he had a human rather than a machine in mind): [[ What I'd prefer to see is that the sheriff the person primarily responsible for reverting broken patches if not fixed in a timely manner. Then we could have some human judgment in the process and specific people with clear responsibility. ]] * https://lists.webkit.org/pipermail/webkit-dev/2010-March/011970.html In which we go through the process manually on webkit-dev again. In summary, there's been a lot of discussion on webkit-dev about how to keep the tree green. Several people (including senior folks in the project) have asked us to build exactly the system we've built. We floated a bunch of approaches and algorithms, got feedback, and took that feedback into account. For example, 1) We didn't just skip the failing/flaky tests. We spent months fixing them. (Ok, we might have skipped one or two, but we and a lot of folks put a lot of effort into fixing them.) 2) Sheriffbot integrates with IRC instead of just with bugs.webkit.org. 3) Sheriffbot doesn't roll out patches automatically. Instead, human judgement is involved. 4) There's a minimum of process (e.g., no pre-set "fix or be rolled out" deadline). 5) There's no notion of a "closed tree". Of course, Eric and I wrote the majority of the code and so got to choose various details of the how things work. If other people are passionate about this topic, we're open to suggestion and contributions. All the code is in svn.webkit.org waiting to be improved. Adam _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
