Thanks for the FS Koushik. I think these enhancements would greatly help in enhancing our test suite, including performance tests. I have few questions/comments:
1. You mention about simulating a fixed fault/ fixed failure. Does this mean that when I configure simulator to fail it always keeps failing for that command until I re-configure it to succeed? Or is there a way I can say, "a command should fail X % of the time"? 2. Can I have different delays configured across different hosts/Clusters for the same command? 3. I believe I can configure a responseType to be a fail/fault and at the same time define a responseJSON as well in tandem with the fault/failure. 4. IIRC, delay is in ms presently. For certain commands like PingCommand, even 1 sec might be a high value. For some of my performance tests, I've used much lesser delay for such commands of the order of ms. 5. Is there an ability to simulate fault for a specific storage/host? For example, I have 5 storages in my cluster but I want to simulate failure for 3 of the storage pools. Is that possible? > -----Original Message----- > From: Koushik Das [mailto:koushik....@citrix.com] > Sent: Tuesday, April 01, 2014 5:48 PM > To: <dev@cloudstack.apache.org> > Subject: [PROPOSAL] Simulator enhancements > > The current simulator provides a way to test various Cloudstack features > without using actual resources. But its current capabilities are limited > since it > only allows success response for all agent commands and some ability to > introduce delays. I am proposing some enhancements to the simulator [1] based > on which test developers can write end-to-end tests for scenarios which were > earlier not possible. Some examples would be VM deployment failure (due to > storage/network failure), VM deployment retry logic, user VM HA etc. > > In the current set of automation tests mostly positive scenarios are tested > as it is > very complex to test negative scenarios with real hardware. With the ability > to > simulate failure, negative scenarios can be tested in an effective manner. > Negative test scenarios will also help in improving the overall code coverage > of > the tests. > > Please share your feedback/comments. > > Thanks, > Koushik > > [1] > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Simulator+enhance > ments
