On Dec 7, 2014 12:24 AM, "Justin Mclean" <jus...@classsoftware.com> wrote: > > Hi, > > > 'Fairly' accurate is not good enough. > > I not done the stats on every release but you would have to agree it's happened many times. The fact we don't have only 1 or 2 RC for each release confirms it. I'd estimate that 80%+ of the releases have had more than 2 RCs I think "fairly accurate" covers that.
I meant your statement about the PMC, not the stats. > > > The point is, by painting with such broad strokes, you are discounting the > > efforts of those who take time to contribute to the project. > > I'm not discounting anyones effort there and I really not know why you would assume or state that. That is what your language suggests. Which is what many of us have been trying to tell you. > > We do have some issue here. For instance: > - How many people have added new mustella tests? > - How many new mustella tests have been added since incubation? > - How many PMC members run the tests themselves on the RC? > - How many committers run the mustella test before checking anything in? > - How many committers/PMC members even know how to run the tests? > - We have regular failures of the mustella test runs - often for no explained reasons > - We need to rerun failed tests several times just to get all the tests to pass - that's hardly ideal > - We've run into mustella tests that are wrong and just working by accident None of what you mentioned are problems with Mustella itself. > > Yes the CI solves some of the later points and I totally agree having tests (of any sort) is better than not having them but it's fair to describe it as an imperfect solution. That is not what you said. You blamed the release process on Mustella. > > > But there is no point in blaming the CI system or the Mustella tests for > > our difficult release process. > > It has caused issues in the past ie Michael + Nick collection changes. Because the changes broke existing tests. What would be different if those were FlexUnit or JUnit tests? Thanks, Om > > Thanks, > Justin
