On 12/09/2013 08:14 PM, Lucas Meneghel Rodrigues wrote: > > So, on a typical virt-test workload, we save about 34 % of the time > needed to run the suite. > > I was talking to Lukas, and we think tests such as virtio_console could > be much more impacted from this. Last time I ran those tests, they > already take up to 3:30 hours. > > Let me come up with more data so we can better think about this.
Yeah, I think the impacts to < 10-minute test set aren't very consequential. It's the 10+ hours of testing we want to know if it's going to become 20+ hours. Either way, I don't think this change makes sense unless we also couple it with a simplification of the default env_process behavior. In other words, I think we need to reduce the default complexity by a quite a bit to help justify the test-time increases. Based on Ypu's feedback, I'm thinking we can break the current functions down into more modular parts. Then let tests make use of them as they need (rather than assume all tests need the same steps). Also I think we should provide some harness introspection data to tests, to help them decide what optimization are "safe". Even if this is only the prior and subsequent cartesian names, it would be really useful. Based on Ademar's feedback, it sounds like defaulting to "ignorance-mode" is best. Use our command line options to allow users to decide which optimization they'd like to include (if any). By default, run every test with a clean-as-can be setup. Anything that touches disks in a significant way will likely have the most impact, so if we focus optional optimization in that area, they will have the biggest benefit. Still, I'm not anticipating any quick solutions here, though I'm happy to see some experimentation happening already. -- Chris Evich, RHCA, RHCE, RHCDS, RHCSS Quality Assurance Engineer e-mail: cevich + `@' + redhat.com o: 1-888-RED-HAT1 x44214 _______________________________________________ Virt-test-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/virt-test-devel
