> -----Original Message----- > From: prasanna [mailto:srivatsav.prasa...@gmail.com] On Behalf Of > Prasanna Santhanam > Sent: Thursday, March 07, 2013 2:53 AM > To: cloudstack-dev@incubator.apache.org > Subject: Re: [PROPOSAL] BVT for CloudStack checkins > > On Thu, Mar 07, 2013 at 01:19:19AM +0530, Frank Zhang wrote: > > > - improve simulator for runtime testability > > > - customize to model and inject failures > > > - make a habit of writing tests around bug reports (I started > > > tagging tests since api_refactoring on JIRA already. > > > look for the integration-test label) > > > - make integration testing easier using factories and DSLs (from > > > Chiradeep) > > > > Sorry for long thread again. All my opinion is "don't distract from > > tools, improving Marvin should be good enough". Below are points that > > you can simply skip. > > Frank, no need to apologize, these are useful insights. My original proposal > was only to refactor the marvin library as you suggest. It still is to > provide a > good library that involves less typing in python. > > > > 2. capability > > DSL varying from common language usually lacks loop, condition > > statement, and sometimes variable. But these things are very important > > to our test. if you strip these abilities from your DSL, I believe you > > will finally add them back that ends up with writing a common > > language, then you have been distracted much from writing test case > > > DSL is a good to have for non-programmers - the target audience could be > those who are not comfortable writing python code and qa engineers who > aren't experienced writing code. Agree that it will have its limitations. The > dsl > is only for quickly writing a few simple tests, it's not to replace the actual > integrated test writing using python. I won't be doing that as it will > convolute > things. > > > > 3. it's not our business now > > The urgent need for CloudStack now is a set of reliable test cases, > > which I think could be done by improving current Marvin. Building our > > DSL distracts us at this point. > > If not for writing tests I"m sure it can be used for describing deployments > for > marvin. I'll figure out a compromise and your inputs on this is most welcome!
Cool. We used to have that but have been obsoleted. This is most valuable thing I see for CloudStack, marvin should use XML to deploy setup. This may be the most widely spread DSL that is comfortable for even non-programmer > > > 4. you don't even need to search a test framework for this time being > > When I studying DSL, I ever looked into some famous test framework. > > For Behavior Driven Test I looked Cucumber(Ruby), for Model Driven > > Test I looked fpmt(the name maybe wrong, written by a French). They > > both use DSL. My conclusion is the benefit of DSL is for stuff that > > has nothing to do with test logic itself. For example, driving test > > cases, producing document/report, scheduling random test case > > combination ... > > I did read enough dsl-bashing on the interwebs when I explored about it > (terms like cargo cults from the ruby era come to mind) but it might be worth > a shot. We won't know until we try. > > > they are nice to have, but not urgent. So for our purpose, I would > > suggest using python nose which is a unit test case framework but can > > still use for integration test. simple, quick, and easy. > > The nose support for marvin has existed for sometime and you can use many > of nose's plugins today including - pdb, attributes, multiprocess etc. > > -- > Prasanna.,
