The patchset adds additional AIO driver abstraction to the block raw driver to
support multiple aio implementations for each device. The first patch pulls
the posix aio implementation out of the block-raw device using a generic call
to the newly created AIO Driver structure. The posix aio implementation that
was contained in block-raw-posix.c has been refactored in to aio-posix.c. The
next patch adds a linux aio implementation for raw devices being opened
O_DIRECT via cache=off drive option. We only use linux aio when cache=off as
linux aio falls back to synchronous ops if not opened with O_DIRECT flag.
Addtional work has been done on top of QEMU for KVM and virtio-blk devices.
While virtio-blk is not yet upstream in QEMU, the AIO changes here provide a
tremendous performance improvement (from 7.6% of native, to 100% of randwrite,
and 3.9% of native, to 101.4% of native for seq write) for virtio
devices with cache=off.
Storage subsystem:
IBM EXP300 - 14 Disk Fiber Expansion, 17G - 15K RPMS
Host: AMD Barcelona, 2 socket, 8G RAM
HBA: QLogic Corp. ISP2312-based 2Gb Fibre Channel to PCI-X HBA (rev 02)
Benchmark[1]:
fio --name=guestrun --filename=/dev/mapper/volumes-fibre \
--rw=randwrite --bs=16k --ioengine=libaio --direct=1 \
--norandommap --runtime=120 --time_based --numjobs=1 \
--group_reporting --thread --size=25g --write_lat_log \
--write_bw_log --iodepth=74
Qemu parameters:
-m 1024 \
-drive file=/images/npt2-guest-virtio.qcow2,if=ide,boot=on,snapshot=off \
-drive file=/dev/mapper/volumes-fibre,if=virtio,cache=(on|off) \
-drive file=/dev/mapper/volumes-npt2--dom1,if=virtio,cache=off
-net nic,macaddr=00:FF:FF:00:00:01,model=rtl8139 -net tap -vnc :123 \
-monitor stdio
Guest io scheduler: noop
Results:
These results are with the patch series applied to KVM (plus a small KVM only
change -- KVM patches forthcoming).
16k randwrite 1 thread, 74 iodepth | MB/s | avg sub lat (us) | avg comp lat
(ms)
---------------------------------------+---------------------+------------------
baremetal (O_DIRECT, aka cache=off)| 61.2 | 13.07 | 19.59
kvm: cache=off posix-aio w/o patch | 4.7 | 3467.44 | 254.08
kvm: cache=off linux-aio | 61.1 | 75.35 | 19.57
kvm: cache=on posix-aio w/o patch |127.0 | 115.78 | 9.19
kvm: cache=on posix-aio w/ patch |126.0 | 67.35 | 9.30
16k write 1 thread, 74 iodepth | MB/s | avg sub lat (us) | avg comp lat
(ms)
---------------------------------------+---------------------+------------------
baremetal (O_DIRECT, aka cache=off)|128.1 | 10.90 | 9.45
kvm: cache=off posix-aio w/o patch | 5.1 | 3152.00 | 231.06
kvm: cache=off linux-aio |130.0 | 83.83 | 8.99
kvm: cache=on posix-aio w/o patch |184.0 | 80.46 | 6.35
kvm: cache=on posix-aio w/ patch |165.0 | 70.90 | 7.09
1. http://brick.kernel.dk/snaps/fio-1.21.tar.bz2
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
