On 02/09/2011 01:42 PM, Blue Swirl wrote:
On Fri, Feb 4, 2011 at 3:49 PM, Michael Roth<mdr...@linux.vnet.ibm.com> wrote:
These patches apply to master (2-04-2011), and can also be obtained from:
git://repo.or.cz/qemu/mdroth.git qtest_v1
OVERVIEW:
QEMU currently lacks a standard means to do targeted unit testing of the device
model. Frameworks like kvm-autotest interact via guest OS, which provide a
highly abstracted interface to the underlying machine, and are susceptable to
bugs in the guest OS itself. This allows for reasonable test coverage of guest
functionality as a whole, but reduces the accuracy and specificity with which
we can exercise paths in the underlying devices.
The following patches provide the basic beginnings of a test framework which
replaces vcpu threads with test threads that interact with the underlying
machine directly, allowing for directed unit/performance testing of individual
devices. Test modules are built directly into the qemu binary, and each module
provides the following interfaces:
init():
Called in place of qemu's normal machine initialization to setup up devices
explicitly. A full machine can be created here by calling into the normal init
path, as well as minimal machines with a select set of buses/devices/IRQ
handlers.
run():
Test logic that interacts with the now-created machine.
cleanup():
Currently unused, but potentially allows for chaining multiple tests
together. Currently we run one module, then exit.
As mentioned these are very early starting points. We're mostly looking for
input from the community on the basic approach and overall requirements for an
acceptable framework. A basic RTC test module is provided as an example.
BUILD/EXAMPLE USAGE:
$ ./configure --target-list=x86_64-softmmu --enable-qtest --enable-io-thread
$ make
$ ./x86_64-softmmu/qemu-system-x86_64 -test ?
Available test modules:
rtc
$ ./x86_64-softmmu/qemu-system-x86_64 -test rtc
../qtest/qtest_rtc.c:test_drift():L94: hz: 2, duration_ms: 4999,
exp_duration: 5000, drift ratio: 0.000200
../qtest/qtest_rtc.c:test_drift():L111: hz: 1024, duration_ms: 4999,
exp_duration: 5000, drift ratio: 0.000200
GENERAL PLAN:
- Provide libraries for common operations like PCI device enumeration, APIC
configuration, default-configured machine setup, interrupt handling, etc.
- Develop tests as machine/target specific, potentially make some tests
re-usable as interfaces are better defined
- Do port i/o via cpu_in/cpu_out commands
- Do guest memory access via a CPUPhysMemoryClient interface
- Allow interrupts to be sent by writing to an FD, detection in test modules
via select()/read()
TODO:
- A means to propagate test returns values to main i/o thread
- Better defined test harness for individual test cases and/or modules,
likely via GLib
- Support for multiple test threads in a single test module for scalability
testing
- Modify vl.c hooks so tests can configure their own timers/clocksources
- More test modules, improve current rtc module
- Further implementing/fleshing out of the overall plan
Comments/feedback are welcome!
Would it be possible to couple this with the tracing or Kemari somehow
so that you could capture, say, block device traces and feed them to
test setup?
I would think so...it's a pretty open ended framework, a unit test
could, say, read in block device traces in some pre-defined format and
then execute those against a block device. We're also planning on adding
command-line parameters for tests, so a unit test could actually be used
as a general testing utility. for instance:
qemu -test block-trace-virtio -test-opts
tracefile=<file>,target_img=<img>,target_fmt=qcow2,comparison_img=<img>
