On Tue, Apr 16, 2013 at 04:30:17PM -0400, Mason Turner wrote:
We have an in-house app, written in c, that is not performing as well as we'd
hoped it would when moving to a VM. We've tried all the common tuning
recommendations (virtio, tap interface, cpu pining), without any change in
performance. Even terminating all of the other VMs on the host doesn't make a
difference. The VM doesn't appear to be CPU, memory or IO bound. We are
trying to maximize UDP-based QPS against the in-house app.
I've been running strace against the app and perf kvm against the VM to try
to identify any bottlenecks. I would say there are a lot of kvm_exits, but
I'm not sure how to quantify what is acceptable and what is not.
We are trying to maximize UDP queries against the app. I've read a few times
that the virtio network stack results in a lot of vm_exits. Unfortunately, we
can't use the direct PCI access with our hardware.
Can you explain the traffic characteristics more?
* UDP packet size
* Pattern: 1 query packet, 1 response packet or something more exotic
* Bare metal QPS (the goal)
* Guest QPS (what you're seeing)
* Benchmark configuration: are packets going across a physical network?
Is there a good resource inefficient system calls? Things that result in
higher than normal kvm_exits, or other performance killers?
Thanks for the help.
Our hypdervisor is running on
CentOS 6.3: 2.6.32-279.22.1.el6.x86_64
qemu-kvm 0.12.1.2
libvirt 0.9.10
Our app is running on
Centos 6.1: 2.6.32-131.0.15.el6.x86_64
Slightly outdated guest and host. It might be worth trying upstream
kernels and QEMU (build from source).
domain type='kvm'
namething1/name
uuidabe76ce9-60a0-4727-a7ae-cf572e5c3f21/uuid
memory unit='KiB'16384000/memory
currentMemory unit='KiB'16384000/currentMemory
vcpu placement='static'6/vcpu
cputune
vcpupin vcpu='0' cpuset='0'/
vcpupin vcpu='1' cpuset='2'/
vcpupin vcpu='2' cpuset='4'/
vcpupin vcpu='3' cpuset='6'/
vcpupin vcpu='4' cpuset='8'/
vcpupin vcpu='5' cpuset='10'/
/cputune
numatune
memory mode='interleave' nodeset='0,2,4,6,8,10'/
/numatune
os
type arch='x86_64' machine='rhel6.0.0'hvm/type
boot dev='hd'/
/os
features
acpi/
apic/
pae/
/features
clock offset='utc'/
on_poweroffdestroy/on_poweroff
on_rebootrestart/on_reboot
on_crashrestart/on_crash
devices
emulator/usr/libexec/qemu-kvm/emulator
disk type='file' device='disk'
driver name='qemu' type='raw' cache='none'/
source file='/var/lib/libvirt/images/thing1-disk0'/
target dev='vda' bus='virtio'/
address type='pci' domain='0x' bus='0x00' slot='0x05'
function='0x0'/
/disk
controller type='usb' index='0'
address type='pci' domain='0x' bus='0x00' slot='0x01'
function='0x2'/
/controller
interface type='bridge'
mac address='00:5e:e3:e1:8a:aa'/
source bridge='virbr0'/
model type='virtio'/
address type='pci' domain='0x' bus='0x00' slot='0x04'
function='0x0'/
/interface
Please double-check that vhost-net is being used:
http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/topic/liaat/liaatbpvhostnet.htm
serial type='pty'
target port='0'/
/serial
console type='pty'
target type='serial' port='0'/
/console
input type='tablet' bus='usb'/
input type='mouse' bus='ps2'/
graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/
video
model type='cirrus' vram='9216' heads='1'/
address type='pci' domain='0x' bus='0x00' slot='0x02'
function='0x0'/
/video
memballoon model='virtio'
address type='pci' domain='0x' bus='0x00' slot='0x06'
function='0x0'/
/memballoon
/devices
/domain
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
