[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
I agree we should not change the behavior too much now that FeatureFreeze is passed. Adjusting tasks in consequence. ** Changed in: qemu-kvm (Ubuntu Maverick) Status: New = Won't Fix ** Changed in: qemu-kvm (Ubuntu) Importance: Undecided = Wishlist ** Changed in: qemu-kvm (Ubuntu) Status: Won't Fix = Triaged ** Summary changed: - Lucid qemu-kvm: ksmd default config is CPU hog + qemu-kvm: ksmd should have a less CPU-hogging default config -- qemu-kvm: ksmd should have a less CPU-hogging default config https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
@Nullzone: that's some very good input, thanks. Looking at your conclusions: ksm is good but need some tweaks to doesnt become a problem more than a help Right, if there is a saner default we should use it, and if there is no sane default we should consider disabling it by default, with strong pointers on how to enable it with the right sleep_millisecs value. I'd consider to off ksm in /etc/default/qemu-kvm for desktop versions of Ubuntu (availabe but off by default) That's difficult to do, since packages have no way to tell if they are on a server. I'd consider to switch default value of sleep_millisecs: 120 even a bit higher in the ubuntu server version. More conservative... +1 on choosing a less aggressive setting. Ideally upstream would also change its default setting. CCing Serge Hallyn since he now owns that stack, for his input. -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
Thanks for the info Nullzone! Based on your data, I'm going to spend a bit of time running with: # To disable qemu-kvm's page merging feature, set KSM_ENABLED=0 and # sudo restart qemu-kvm KSM_ENABLED=1 #SLEEP_MILLISECS=2000 SLEEP_MILLISECS=50 PAGES_TO_SCAN=100 in my /etc/default/qemu-kvm, and if [ -w /sys/kernel/mm/ksm/pages_to_scan ]; then if [ -n $PAGES_TO_SCAN ]; then echo $PAGES_TO_SCAN /sys/kernel/mm/ksm/pages_to_scan fi fi added to my /etc/init/qemu-kvm.conf I think upping sleep_millisec for Natty is a good idea. But not for Lucid+Maverick, it's just too late, and after all these are trivially changeable and really ought to be customized anyway. I also think (obviously) that adding KSM_PAGES_TO_SCAN to the list of variables which /etc/init/qemu-kvm can read seems worthwhile. However again I'm not sure whether I want to ask for a feature freeze exception for maverick for this. I personally think it's better to wait for Natty. Thierry, do you concur, or should I try for maverick? -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
Hello Thierry, I will try to help showing some information gathered from my lab server as you requested: - Enviroment: 10 x Ubuntu 10.04 x86_64 with several services (SMTP, MySQL, Oracle, DB2, TeamSpeak servers, Tomcats, bind ... etc). 1 x Windows 7 x86_64 3 x Debian Lenny x86_64 (iscsitarget, pacemaker cluster testing) RAM schema (assigned to the VMs): 3 vms 128 MB, 6 vms 256 MB, 2 vms 512 MB, 1 vm 768 MB, 2 vms 1GB SMP: 5 vms with 2 SMP (num vCPU) configured, rest 1 SMP - Hardware: 1 x Intel(R) Core(TM)2 Quad CPU @ 2.66GHz 8 GB RAM DDR2 800 Mhz / 25 ns - * Summary: Usefull information: CPU % usage goes stable, also memory is examined and released after the 3rd full_scans CPU % usage is specially high between 1st and 2nd scan (when ksm is merging pages at the same time that it is doing the second scan) Memory starts being released after the 1st scan is completly and the 2nd is in progress (so past some minutes) KSM default pages_to_scan:100 sleep_millisecs:20 CPU: ~11,67% Time_for_a_full_scan: 3m35.209s KSM pages_to_scan:60 sleep_millisecs:40 CPU: ~3,44% Time_for_a_full_scan: 11m50.734s KSM pages_to_scan:100 sleep_millisecs:50 CPU: ~4,21% Time_for_a_full_scan: 8m55.533s KSM pages_to_scan:60 sleep_millisecs:60 CPU: ~2,00% Time_for_a_full_scan: 17m46.094s KSM pages_to_scan:100 sleep_millisecs:120 CPU: ~1,67% Time_for_a_full_scan: 21m26.285s Mem Optimization in all the tests (there before or after): RAM available (reminder): 8 GB Memory optimizacition thanks to ksm: 14-16% full available memory (~ 1.3 / 1.1 Gb of more free memory) Conclusions: ksm is good but need some tweaks to doesnt become a problem more than a help I'd consider to off ksm in /etc/default/qemu-kvm for desktop versions of Ubuntu (availabe but off by default) I'd consider to switch default value of sleep_millisecs: 120 even a bit higher in the ubuntu server version. More conservative... --- r...@core:/sys/kernel/mm/ksm# echo 2 /sys/kernel/mm/ksm/run r...@core:/sys/kernel/mm/ksm# sleep 30 r...@core:/sys/kernel/mm/ksm# landscape-sysinfo System load: 1.02 IP address for lo: xxx Usage of /home: 66.1% of 2.68TB IP address for eth1: xxx Memory usage:60% IP address for eth1:0: xxx Swap usage: 0%IP address for bond0: xxx Processes: 199 IP address for br0:xxx Users logged in: 1 r...@core:/sys/kernel/mm/ksm# echo 1 /sys/kernel/mm/ksm/run ; time /tmp/tmp.bash real3m35.209s user0m0.070s sys 0m0.130s [... /tmp/tmp.bash #!/bin/bash while true do SALIDA=`cat /sys/kernel/mm/ksm/full_scans` if [ $SALIDA == 1 ] then break else sleep 5 fi done yeee... terrible but enough for a script done in 10 secs...] [...waiting for a pair of aditional scans...] r...@core:/sys/kernel/mm/ksm# cat full_scans 3 r...@core:/sys/kernel/mm/ksm# ps axuw | grep ksm | grep -v grep root53 10.7 0.0 0 0 ?SN 17:51 12:08 \_ [ksmd] (Cycle/scan time is 3m35secs ~= 215 secs -- doing check each 2 secs so I'll use iostat -p pid 2 215/2) r...@core:/sys/kernel/mm/ksm# pidstat -p 53 2 108 Linux 2.6.32-24-server (core) 18/08/10_x86_64_(4 CPU) [...] Media: 530,00 11,670,00 11,67 - ksmd r...@core:/sys/kernel/mm/ksm# cat full_scans 4 r...@core:/sys/kernel/mm/ksm# cat max_kernel_pages 511518 r...@core:/sys/kernel/mm/ksm# cat pages_shared 42045 r...@core:/sys/kernel/mm/ksm# cat pages_sharing 287118 r...@core:/sys/kernel/mm/ksm# cat pages_to_scan 100 r...@core:/sys/kernel/mm/ksm# cat pages_unshared 622240 r...@core:/sys/kernel/mm/ksm# cat pages_volatile 97214 r...@core:/sys/kernel/mm/ksm# cat run 1 r...@core:/sys/kernel/mm/ksm# cat sleep_millisecs 20 r...@core:/sys/kernel/mm/ksm# cat /proc/meminfo MemTotal:8194212 kB MemFree: 1000880 kB Buffers: 432396 kB Cached: 2865740 kB SwapCached:0 kB [...] r...@core:/sys/kernel/mm/ksm# landscape-sysinfo System load: 6.44 IP address for lo: xxx Usage of /home: 66.1% of 2.68TB IP address for eth1: xxx Memory usage:44% IP address for eth1:0: xxx Swap usage: 0%IP address for bond0: xxx Processes: 199 IP address for br0:xxx Users logged in: 1 r...@core:/sys/kernel/mm/ksm# echo 2 /sys/kernel/mm/ksm/run r...@core:/sys/kernel/mm/ksm# sleep 30 r...@core:/sys/kernel/mm/ksm# echo 40 /sys/kernel/mm/ksm/sleep_millisecs r...@core:/sys/kernel/mm/ksm# echo 60 /sys/kernel/mm/ksm/pages_to_scan r...@core:/sys/kernel/mm/ksm# echo 1 /sys/kernel/mm/ksm/run ; time
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
17% sounds a bit strong... I'd very much like to see a memory gain / CPU usage chart for various values, or more discussion around the acceptable value upstream. Any idea what the other qemu-kvm distributions are using as a default ? ** Also affects: qemu-kvm (Ubuntu Maverick) Importance: Undecided Status: Won't Fix ** Changed in: qemu-kvm (Ubuntu Maverick) Status: Won't Fix = New -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
This does seem to be a valid bug, some serious CPU time is being dedicated to ksmd. Whilst I haven't seen it in Lucid, i have experienced it in Maverick. Smoser witnessed ~17% CPU utilization for ksmd. Nominating for Maverick. -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
As stated above, I personally don't plan on fixing it in Ubuntu. You can set that value to whatever you want in the config file. Dustin -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
Is this going to be fixed? -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
I don't think Won't Fix is appropriate. If you look at at kernel/Documentation/vm/ksm.txt you find that the values for pages_to_scan and sleep_millisecs are documentated as chosen for demonstration purposes. So upstream obviously does not consider these values appropriate for normal operation. Therefore I think ksm should either be distributed as disabled or with a less aggressive configuration. -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
See also https://bugzilla.redhat.com/show_bug.cgi?id=541230 ** Bug watch added: Red Hat Bugzilla #541230 https://bugzilla.redhat.com/show_bug.cgi?id=541230 -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 578930] Re: Lucid qemu-kvm: ksmd default config is CPU hog
Hi there- You can actually set it to whatever you want, and have it persist across boots. Just modify the values in the file /etc/default/qemu-kvm. We're just using the default, upstream kernel value for SLEEP_MILLISECONDS, which is 20. It does seem a bit aggressive, but it's the upstream default. We should have a discussion with them to see if it makes sense to change it. However, I'm going to mark this won't-fix against the qemu-kvm package, since we have an easy configuration file where you can tune this value to your liking. ** Changed in: qemu-kvm (Ubuntu) Status: New = Won't Fix -- Lucid qemu-kvm: ksmd default config is CPU hog https://bugs.launchpad.net/bugs/578930 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
