On 12/05/2013 04:22 PM, Ademar de Souza Reis Jr. wrote:
> Do you have any numbers on the amount of time saved on a typical
> run of virt-test with/without these optimizations? In other
> words, what's the magnitude of the problem we're solving by
> turning these optimizations on by default?
>
> Thanks.
> - Ademar
Aye, we default to "optimized" currently, and I agree with your
position. LMR and I attempted to consider more rationally, but our data
is quite limited and biased:
* Internal grid and by-hand runs of select qemu-kvm tests show around
50% time savings.
* By-hand runs of select libvirt tests show roughly 33% time savings.
* No data from un-selected tests, or those requiring special host setup.
* Subjective time/cost impact from:
* Additional maintenance/bugfixes related to optimization.
* Additional maintenance related to handling/miss-handling false-
positives and silent failures (both have been encountered
recently)
* Subjective cost of unreliable/questionable results over the long-term
(years). e.g. was that disk test Failure 3 years ago related to
some current problem, or suspect to an optimization
influenced indeterminable condition?
Hence why I'd feel more comfortable with differing data/opinions from
the community. Also, to be fully transparent, I'm personally excited
about the possibility to delete a lot of "magic" code, enhance long-term
result trust, and reduce ongoing maintenance efforts. However, I'll
graciously abandon that position if I'm wrong.
Please share your use-case and info!
--
Chris Evich, RHCA, RHCE, RHCDS, RHCSS
Quality Assurance Engineer
e-mail: cevich + `@' + redhat.com o: 1-888-RED-HAT1 x44214
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
