Hi All.
I've noticed that there is a huge (more than ~3x slower) performance
difference between KVM guest and host machine.
Host machine:
dell r720xd
RAID10 with 12 SAS 15 k drives and RAID0 with 2*128 GB INTEL SSD drives in
Dell CacheCade mode.
*On the KVM guest:*
/usr/pgsql-9.4/bin/pg_test_fsync -f test.sync
5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.
Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
open_datasync 5190.279 ops/sec 193 usecs/op
fdatasync 4022.553 ops/sec 249 usecs/op
fsync 3069.069 ops/sec 326 usecs/op
fsync_writethrough n/a
open_sync 4892.348 ops/sec 204 usecs/op
Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
open_datasync 2406.577 ops/sec 416 usecs/op
fdatasync 4309.413 ops/sec 232 usecs/op
fsync 3518.844 ops/sec 284 usecs/op
fsync_writethrough n/a
open_sync 1159.604 ops/sec 862 usecs/op
Compare open_sync with different write sizes:
(This is designed to compare the cost of writing 16kB
in different write open_sync sizes.)
1 * 16kB open_sync write 3700.689 ops/sec 270 usecs/op
2 * 8kB open_sync writes 2581.405 ops/sec 387 usecs/op
4 * 4kB open_sync writes 1318.871 ops/sec 758 usecs/op
8 * 2kB open_sync writes 698.640 ops/sec 1431 usecs/op
16 * 1kB open_sync writes 262.506 ops/sec 3809 usecs/op
Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written
on a different descriptor.)
write, fsync, close 3071.141 ops/sec 326 usecs/op
write, close, fsync 3303.946 ops/sec 303 usecs/op
Non-Sync'ed 8kB writes:
write 251321.188 ops/sec 4 usecs/op
*On the host machine:*
/usr/pgsql-9.4/bin/pg_test_fsync -f test.sync
5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.
Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
open_datasync 11364.136 ops/sec 88 usecs/op
fdatasync 12352.160 ops/sec 81 usecs/op
fsync 9833.745 ops/sec 102 usecs/op
fsync_writethrough n/a
open_sync 14938.531 ops/sec 67 usecs/op
Compare file sync methods using two 8kB writes:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
open_datasync 7703.471 ops/sec 130 usecs/op
fdatasync 11494.492 ops/sec 87 usecs/op
fsync 9029.837 ops/sec 111 usecs/op
fsync_writethrough n/a
open_sync 6504.138 ops/sec 154 usecs/op
Compare open_sync with different write sizes:
(This is designed to compare the cost of writing 16kB
in different write open_sync sizes.)
1 * 16kB open_sync write 14113.912 ops/sec 71 usecs/op
2 * 8kB open_sync writes 7843.234 ops/sec 127 usecs/op
4 * 4kB open_sync writes 3995.702 ops/sec 250 usecs/op
8 * 2kB open_sync writes 1788.979 ops/sec 559 usecs/op
16 * 1kB open_sync writes 937.177 ops/sec 1067 usecs/op
Test if fsync on non-write file descriptor is honored:
(If the times are similar, fsync() can sync data written
on a different descriptor.)
write, fsync, close 10144.280 ops/sec 99 usecs/op
write, close, fsync 8378.558 ops/sec 119 usecs/op
Non-Sync'ed 8kB writes:
write 159176.122 ops/sec 6 usecs/op
The file system "inside" and "outside" the same - ext4 on LVM. Disk
scheduler "inside" and "outside" set to "noop". Fstab options same to,
setted to rw,noatime,nodiratime,barrier=0. OS on host and guest the same
CentOS release 6.5 (Final).
Guest volume options:
Disk bus: Virtio
Cache mode: none
IO mode: native
Any ideas?
