On Wed, Aug 08, 2012 at 12:27:40PM +0100, Phil Holmes wrote: > We're getting to the time where the Grand Regression Test Checking > (TM) project > (http://lilypond.org/doc/v2.15/Documentation/contributor/grand-regression-test-checking > ) has got enough results to allow us to use it to fix poor regtests. > We therefore need to agree how to get agreement on the changes > needed to the regtests. To put it into perspective, there are over > 1000 regtests, of which 200 have been shown to need attention. My > thoughts on our options:
What does "need attention" mean? My blind guess is that half of those are simply typos / grammar / clarifying the text. Then half of the remainder need fairly simple and unproblematic changes to the lilypond input. Finally, half of the remainder will require significant discussion. > New Google Project > ============== > This occurred to me only recently. What about setting up a new > Google Project called "LilyPond Regression Test Improvement". This > would require a new mailing list to avoid the problems above. We > could then add an issue per regtest, and only people who want to > subscribe would need to subscribe to the list. > > I think the final suggestion is my favoured one. I don't favor that option. Let's look at this backwards: for every single change to lilypond git, we need a patch on rietveld submitted via git-cl. In some cases (e.g., fixing typos) you might get a review within an hour saying "LGTM, please push directly to staging". But any non-totally-trivial change is going to require a full countdown, comments, etc. I don't think we should be handing out blank cheques to anybody with regards to the regtests. I'm not comfortable letting any "regtest improvement team" send changes directly to git without being on a general countdown. Who wants to be producing these patches? I recall warning about this situation before the whole checker started up, and I thought that there was a volunteer to handle it. Situations change, of course, but it's a bit disappointing to be discussing how to handle the feedback now. None of the suggestions make the patch-production easier. They all center around discussing what types of changes to make, which is certainly a necessary first step, but they leave the question of creating patches unanswered. My suggestion: make the data available, then sit back and see who wants to deal with it. Maybe add another column to the spreadsheet or database for "regtest updated", then that can be filled in as it happens. Basically, let people handle it manually. I don't think that we're going to have a deluge of developers interested in working on this, so I don't think that we'll have synchronization problems from multi-threading. A global lock (i.e. person X is working on the feedback; person Y waits until person X is finished) should suffice. - Graham _______________________________________________ lilypond-devel mailing list [email protected] https://lists.gnu.org/mailman/listinfo/lilypond-devel
