On Wed, Nov 21, 2012 at 6:29 PM, Animesh Chaturvedi < animesh.chaturv...@citrix.com> wrote:
> David > > I certainly vote +1 on your ideas to improve testing in CS and enhancing > and embracing automation as way of life. To put in my two cents , I would > like to highlight that unit test automation is the base for automation > strategy. Here is a good post on test pyramid > http://martinfowler.com/bliki/TestPyramid.html. As developers if we can > commit to comprehensive automated unit tests we can release early and > often. CloudStack with its API layer is already well positioned for > service/integration layer automation. > > I agree with Martin's sentiment there. Really what we want to do is get a pipeline going. (and actually now that I write this, I realize that I need to adjust how things are evolving on b.a.o) so that it looks like this: * RAT * compile + junit * docs + apidocs * package building * install tests * smoketests * full blown integration tests A failure at any stage should block every other future test. This also gets us from least expensive (RAT completes in around 1 minute) to most expensive (the full Marvin tests take hours) Running this frequently also allows us to quickly see where problems have been introduced because there would be a limited set of changes in each of these test iterations. --David
