On Mon, May 12, 2008 at 02:19:24PM -0500, Ryan Harper wrote: > I've been digging into some of the instability we see when running > larger numbers of guests at the same time. The test I'm currently using > involves launching 64 1vcpu guests on an 8-way AMD box. With the latest > kvm-userspace git and kvm.git + Gerd's kvmclock fixes, I can launch all > 64 of these 1 second apart, and only a handful (1 to 3) end up not > making it up. In dmesg on the host, I get a couple messages: > > [321365.362534] vcpu not ready for apic_round_robin > > and > > [321503.023788] Unsupported delivery mode 7 > > Now, the interesting bit for me was when I used numactl to pin the guest > to a processor, all of the guests come up with no issues at all. As I > looked into it, it means that we're not running any of the vcpu > migration code which on svm is comprised of tsc_offset recalibration and > apic migration, and on vmx, a little more per-vcpu work
Hi Ryan, There are two places that attempt to use delivery mode 7: kexec crash and io_apic_64.c::check_timer(). The later will happen if the guest fails to receive PIT IRQ's for 10 ticks. If you're using HZ=1000 thats 10ms. See timer_irq_works(). The in-kernel pit emulation has logic which avoids injecting more than one IRQ during 10ms. Note that the guest 10ms delay is TSC based and uses only the lower 32-bits of the value. It is quite likely that the TSC adjustment results in them increasing more rapidly then they should. So can you try setting KVM_MAX_PIT_INTR_INTERVAL to a lower value? HZ/10 or something. You can confirm this theory by booting the guests with "apic=debug". ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
