On Wed, Nov 21, 2012 at 6:38 PM, David Nalley <da...@gnsa.us> wrote: > On Wed, Nov 21, 2012 at 6:29 PM, Animesh Chaturvedi < > animesh.chaturv...@citrix.com> wrote: > >> David >> >> I certainly vote +1 on your ideas to improve testing in CS and enhancing >> and embracing automation as way of life. To put in my two cents , I would >> like to highlight that unit test automation is the base for automation >> strategy. Here is a good post on test pyramid >> http://martinfowler.com/bliki/TestPyramid.html. As developers if we can >> commit to comprehensive automated unit tests we can release early and >> often. CloudStack with its API layer is already well positioned for >> service/integration layer automation. >> >> > I agree with Martin's sentiment there. > Really what we want to do is get a pipeline going. (and actually now that I > write this, I realize that I need to adjust how things are evolving on > b.a.o) so that it looks like this: > * RAT > * compile + junit > * docs + apidocs > * package building > * install tests > * smoketests > * full blown integration tests > > A failure at any stage should block every other future test. This also gets > us from least expensive (RAT completes in around 1 minute) to most > expensive (the full Marvin tests take hours) Running this frequently also > allows us to quickly see where problems have been introduced because there > would be a limited set of changes in each of these test iterations. > > --David
+1 to that order
