* On 2013-03-04 15:20:49 -0500, Chris Evich ([email protected]) wrote: > On 02/20/2013 07:52 AM, Qingtang Zhou wrote: > >Hi, guys, > > > >I find when qemu get crashed, the standalone runner still reports > >test is PASS. This incorrect report would make us miss some > >important failure. > > Did anyone ever respond to this bug? I found this problem in normal autotest workflow (run autotest via control file), and reported in irc. > > >My idea is re-format our env_process workflow, putting all guest > >related operation, such as stop, cont, ..., into test script script > >scope, in this case our crash detection mechanism can take their > >responsibility. > > I've always though the "env_process for everyone!" assumption was > problematic. It's lead to way too many special-case parameters and > a very complicated set of conditionals. I'd really like to see a > more object-oriented approach with the test objects inheriting a set > of pre-and-post env-handler behaviors. Me too > > In this way, individual test-modules could then override (or delete) > methods which are irrelevant (or unwanted), or do nothing and get a > default set of behaviors. There was even an issue opened on this a > while ago: > > https://github.com/autotest/virt-test/issues/26 > > One big impediment is, the new test runner uses > virttest.standalone_test.Test (a faceless Object class) whereas > autotest uses virt.virt (a test.test subclass). Otherwise we could > easily hook in custom pre-and-post processing behavior via the > existing test.test.initialize(), setup(), and > postprocess_iteration() methods :_( > > Probably the only other way this could be done is by heaping more > code into both standalone_test.Test.run_once() and > virt.virt.run_once(). Since this is the head of most things > env_process related. It's not as "pretty" but it may be better then > further complicating the env_process conditionals beyond > comprehension. I'd like to divide the existed env_process codes into some pieces: * A base cass, providing a series of interfaces * Some sub classes for every test type (qemu/libvirt/v2v/...) to implement these interfaces. Then test cases should call these interfaces inside their run_xxx functions, instead of calling them in framework (I mean, the huge try...try...try...except...except block) Well, this idea is under discussion internally.
Yeah, your idea is great and could do least modification in the test script, but I doubt the *.run_once() could become another place where is full of conditional branches. > > >And another problem is the crash handler set by autotest is gone if > >I use the standalone runner, I guess we should bring it back again. > > I vaguely remember lmr explaining the reasoning behind this as: If > you're using the runner, you'll likely notice a crash and don't need > the code to do the work of your eyeballs. I could be mistaken > though. Though you may don't want to make the test script detect the crash, I think it's necessary to generate crash coredump during testing process. I guess you don't like to manually run qemu and get the core file as well. _______________________________________________ Virt-test-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/virt-test-devel
