I think the model we are following with functionality regressions works
well. We are not necessarily preventing future checkins unless things
are perceived to be Out of Hand by one or more committers.
I think the same could be done for code coverage regressions. If a new
chunk of code is checked in and the numbers drop way way down for a
given module, I think that is cause for concern and a committer could
reasonably insist that the feature is not sufficiently tested and back
out the change, or at least block any future checkins in that area until
enough tests are provided.
So I propose having a flag raised when numbers go say 20% below current
baseline for a given package, and then it is up to the committers to
decide what action is necessary.
David
Kathey Marsden wrote:
Rick Hillegas wrote:
In previous lives, I've seen code-coverage metrics generated on, say,
a monthly basis and used as a release barrier. I do not think they are
appropriate as a barrier to checkin.
I think since we seem to be going for very long release cycles instead
of the release early, release often model, release barriers are not very
effective for maintaining quality on the trunk. Also in open source,
developers come and go so the wait until release model doesn't tend to
work that well in that regard either. If Linux could release a new
kernel twice a day, I tend to think that the trunk should be "ready
for release" at any time for completed functionality and even
incremental functionality could be covered.
Kathey
