On 12/08/2013 10:46 PM, Yiqiao Pu wrote: > Another use of this policy is making it easier to compose the simply > cases together to finish a more complex test scenario. Very useful for > some long term testing. Like in longevity testing we need all tests use > the same environment and keep guest always alive. Actually keeping using > a kind of dirty env is more useful in test for finding bugs. The real > env from customer can not always be clean.
Ahh, interesting, so we have some tests which _must_ maintain environment, always. Is that for the _entire_ test job, or just some collection of tests within a larger test job? > > And for the env setup did in pre/post process, we can set up parameters > that can be checked also in pre/post process. But for the operates > inside test, I think it is eaiser to let the tests themselves to try to > restore the status that might influence other cases. It is not hard to The challenge we face here is tests have no insight into the test-list, so they cannot know what test is coming next. We could possibly make this information available, however I think this would just be moving a lot of complexity out of pre/post, and duplicating it many times in tests instead. Either way, I think you are right about letting tests decide what to do, rather then have harness try to be "smart". However, if we take an 'always restore between tests' stance by default, it's less code for tests AND less code for pre/post. For the longevity testing specifically, that I imagine having a special-case flag/param/option where pre/post simply do nothing (or very minimal checking only), and the test can maintain the env in this case. > outside our guest and may need be used more than one case. And another > one is some special operate inside guests which should be recovered > immediately. Sure, and for these special-cases I agree it would be better to handle them inside the tests. If many tests need do same pre/post, they can always share common utils.test code, or import code from other test module. Combine this with 'do nothing' signal to harness, and I am hopeful we can reduce harness complexity but not take away power from tests. Are there other testing situations that would not be addressed by having just harness 'always restore', and harness 'do nothing' modes? -- Chris Evich, RHCA, RHCE, RHCDS, RHCSS Quality Assurance Engineer e-mail: cevich + `@' + redhat.com o: 1-888-RED-HAT1 x44214 _______________________________________________ Virt-test-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/virt-test-devel
