On Nov 2, 2010, at 9:59 AM, Deryck Hodge wrote:
> What do you all think?
I think your argument has some merit, and I very much appreciate the reasoned
way you approached it.
Personally, though, I disagree with the proposal to throw the tests out.
* Regression tests pay back over months and years, not suddenly.
* I am skeptical of the idea of "temporarily" or experimentally throwing them
out.
- I believe that without maintenance, they will die.
- I don't believe that the proposed metrics ("regression" tags in bugs after
tossing out tests for two or three months) for evaluating the experiment convey
their value.
* While Windmill pain has been recurring, the proposal comes right now because
of immediate pain that is caused by unusually large changes to the way tests
are run. Maris is trying to address that pain (while he simultaneously has
other, more scheduled irons in the fire).
Because we don't have a metric for measuring the tests' value, nor a way to
quantitatively compare their value to their cost, this is a matter of opinion.
This is mine.
I've wanted to put Windmill on buildbot/Hudson, separately from the test suite
and more as something that blocks rolling out, rather than landing. Perhaps
running them in isolation like that would reduce their fragility. I don't want
the results to be ignored.
Gary
_______________________________________________
Mailing list: https://launchpad.net/~launchpad-dev
Post to : [email protected]
Unsubscribe : https://launchpad.net/~launchpad-dev
More help : https://help.launchpad.net/ListHelp
