Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/19/2011 07:59 PM, Amit Shah wrote: On (Mon) 19 Dec 2011 [14:59:36], Avi Kivity wrote: On 12/19/2011 02:52 PM, Amit Shah wrote: (snip) S4 needs some treatment, though, as resume after s4 doesn't work with kvmclock enabled. I didn't realise this series was only handling the soft lockup case. What's the issue there? Not sure yet; after resume, the VM freezes while bringing the 2nd vcpu up. Maybe it's just sitting there spinning, maybe the delta needs adjusting. Haven't poked at it yet. Bug 694801 - Guest fail to resume from S4 if guest using kvmclock -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/19/2011 07:50 PM, Marcelo Tosatti wrote: Maybe it is good (not sure), need to look into schedstats and think of cases that would break legitimate guest hangs. And it probably also affects the position of clearing the flag on the guest side as its currently done in Eric's patchset. Is the task going to be TASK_UNINTERRUPTIBLE with SIGSTOP? Don't think so. No. Note Preemption while TASK_RUNNING or TASK_UNINTERRUPTIBLE is not functional for guest-paused-via-QEMU-monitor case. Right, it's a different case. Note SIGCONT can be trapped, so it's similar to monitor cont. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On (Thu) 15 Dec 2011 [13:55:15], Avi Kivity wrote: On 12/08/2011 01:34 PM, Amit Shah wrote: On (Mon) 05 Dec 2011 [15:18:59], Eric B Munson wrote: When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. Guest S4 would need similar treatment, and I think the code in the two approaches can be shared. Just something to consider. Why does S4 need any treatment? The guest is aware that it's sleeping, unlike the other cases treated here. Er, right. S4 needs some treatment, though, as resume after s4 doesn't work with kvmclock enabled. I didn't realise this series was only handling the soft lockup case. Amit -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/16/2011 11:31 AM, Marcelo Tosatti wrote: Slow enough that progress of the watchdog thread is unable to keep up with timer interrupt processing. This is considered a hang and should be reported. It's not a guest hang though! No, but your host system is in such a load state that for the sake of system usability you better print out a warning message. What's the point in printing it in the guest? The guest can't observe host conditions. I don't see the advantage of preempt notifiers over the simple, paravirt solution proposed? Note kvmclock is already paravirt. Right. What do you want to be done in preempt notifiers? Measure what to consider setting this flag? Preemption while TASK_RUNNING or TASK_UNINTERRUPTIBLE. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/19/2011 02:52 PM, Amit Shah wrote: On (Thu) 15 Dec 2011 [13:55:15], Avi Kivity wrote: On 12/08/2011 01:34 PM, Amit Shah wrote: On (Mon) 05 Dec 2011 [15:18:59], Eric B Munson wrote: When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. Guest S4 would need similar treatment, and I think the code in the two approaches can be shared. Just something to consider. Why does S4 need any treatment? The guest is aware that it's sleeping, unlike the other cases treated here. Er, right. S4 needs some treatment, though, as resume after s4 doesn't work with kvmclock enabled. I didn't realise this series was only handling the soft lockup case. What's the issue there? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/15/2011 08:53 PM, Eric B Munson wrote: I am working on V7 to incorporate the __this_cpu_and suggestion, would you consider that for inclusion and we can continue discussing the need for the preemption notification work? Okay. I think that having a guest complain when the host is so loaded that the guest watchdog threads can't make any progress is desirable behavior, it may be the only notification that an admin gets that a particular host is over loaded. It's the wrong admin, consider a cloud scenario where the host admin and guest admin are different people. Notification on host overload (to initiate migration, say) is a good idea, but it would be a different mechanism. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On (Mon) 19 Dec 2011 [14:59:36], Avi Kivity wrote: On 12/19/2011 02:52 PM, Amit Shah wrote: (snip) S4 needs some treatment, though, as resume after s4 doesn't work with kvmclock enabled. I didn't realise this series was only handling the soft lockup case. What's the issue there? Not sure yet; after resume, the VM freezes while bringing the 2nd vcpu up. Maybe it's just sitting there spinning, maybe the delta needs adjusting. Haven't poked at it yet. Amit -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Mon, Dec 19, 2011 at 02:11:08PM -0200, Marcelo Tosatti wrote: On Mon, Dec 19, 2011 at 02:59:10PM +0200, Avi Kivity wrote: On 12/16/2011 11:31 AM, Marcelo Tosatti wrote: Slow enough that progress of the watchdog thread is unable to keep up with timer interrupt processing. This is considered a hang and should be reported. It's not a guest hang though! No, but your host system is in such a load state that for the sake of system usability you better print out a warning message. What's the point in printing it in the guest? The guest can't observe host conditions. I don't see the advantage of preempt notifiers over the simple, paravirt solution proposed? Note kvmclock is already paravirt. Right. What do you want to be done in preempt notifiers? Measure what to consider setting this flag? Preemption while TASK_RUNNING or TASK_UNINTERRUPTIBLE. Maybe it is good (not sure), need to look into schedstats and think of cases that would break legitimate guest hangs. And it probably also affects the position of clearing the flag on the guest side as its currently done in Eric's patchset. Is the task going to be TASK_UNINTERRUPTIBLE with SIGSTOP? Don't think so. Note Preemption while TASK_RUNNING or TASK_UNINTERRUPTIBLE is not functional for guest-paused-via-QEMU-monitor case. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Mon, Dec 19, 2011 at 02:59:10PM +0200, Avi Kivity wrote: On 12/16/2011 11:31 AM, Marcelo Tosatti wrote: Slow enough that progress of the watchdog thread is unable to keep up with timer interrupt processing. This is considered a hang and should be reported. It's not a guest hang though! No, but your host system is in such a load state that for the sake of system usability you better print out a warning message. What's the point in printing it in the guest? The guest can't observe host conditions. I don't see the advantage of preempt notifiers over the simple, paravirt solution proposed? Note kvmclock is already paravirt. Right. What do you want to be done in preempt notifiers? Measure what to consider setting this flag? Preemption while TASK_RUNNING or TASK_UNINTERRUPTIBLE. Maybe it is good (not sure), need to look into schedstats and think of cases that would break legitimate guest hangs. And it probably also affects the position of clearing the flag on the guest side as its currently done in Eric's patchset. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Thu, Dec 15, 2011 at 12:21:16PM +0200, Avi Kivity wrote: On 12/14/2011 08:21 PM, Marcelo Tosatti wrote: On Wed, Dec 14, 2011 at 04:39:56PM +0200, Avi Kivity wrote: On 12/14/2011 02:16 PM, Marcelo Tosatti wrote: Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Long scheduling delays should be considered hangups from the guest perspective. Why? To the guest it looks like slow hardware, but it will interpret it as a softlockup. Slow enough that progress of the watchdog thread is unable to keep up with timer interrupt processing. This is considered a hang and should be reported. It's not a guest hang though! No, but your host system is in such a load state that for the sake of system usability you better print out a warning message. I don't see the advantage of preempt notifiers over the simple, paravirt solution proposed? Note kvmclock is already paravirt. What do you want to be done in preempt notifiers? Measure what to consider setting this flag? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/14/2011 07:58 PM, Eric B Munson wrote: I don't think you should see a guest softlockup if the host is overloaded. Nor should you see it due to a long live migration pause, or STOP/CONT. You should see a guest softlockup if it is spinning due to a guest bug, and not for any other reason. I think we need a mix of the ioctl (for STOP/CONT and voluntary pauses, like live migration or qemu stop/cont) and preempt notifiers (together with a check for TASK_RUNNING/TASK_UNINTERRUPTIBLE, for scheduling or swap delays) Do you want the preemption notifier work in before you will consider merging this set, or can that be a follow on? Yes please. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/14/2011 08:21 PM, Marcelo Tosatti wrote: On Wed, Dec 14, 2011 at 04:39:56PM +0200, Avi Kivity wrote: On 12/14/2011 02:16 PM, Marcelo Tosatti wrote: Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Long scheduling delays should be considered hangups from the guest perspective. Why? To the guest it looks like slow hardware, but it will interpret it as a softlockup. Slow enough that progress of the watchdog thread is unable to keep up with timer interrupt processing. This is considered a hang and should be reported. It's not a guest hang though! -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/08/2011 01:34 PM, Amit Shah wrote: On (Mon) 05 Dec 2011 [15:18:59], Eric B Munson wrote: When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. Guest S4 would need similar treatment, and I think the code in the two approaches can be shared. Just something to consider. Why does S4 need any treatment? The guest is aware that it's sleeping, unlike the other cases treated here. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Thu, 15 Dec 2011, Avi Kivity wrote: On 12/14/2011 07:58 PM, Eric B Munson wrote: I don't think you should see a guest softlockup if the host is overloaded. Nor should you see it due to a long live migration pause, or STOP/CONT. You should see a guest softlockup if it is spinning due to a guest bug, and not for any other reason. I think we need a mix of the ioctl (for STOP/CONT and voluntary pauses, like live migration or qemu stop/cont) and preempt notifiers (together with a check for TASK_RUNNING/TASK_UNINTERRUPTIBLE, for scheduling or swap delays) Do you want the preemption notifier work in before you will consider merging this set, or can that be a follow on? Yes please. -- error compiling committee.c: too many arguments to function I am working on V7 to incorporate the __this_cpu_and suggestion, would you consider that for inclusion and we can continue discussing the need for the preemption notification work? I think that having a guest complain when the host is so loaded that the guest watchdog threads can't make any progress is desirable behavior, it may be the only notification that an admin gets that a particular host is over loaded. Eric signature.asc Description: Digital signature
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Sun, Dec 11, 2011 at 02:40:58PM +0200, Dor Laor wrote: When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Guests need to run NTP regardless, not only the virtualization layer add some skew, the physical world is not that perfect. btw: traditional NTP client won't sync the time automatically if the diff is 0.5%. Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Isn't it solved by steal time? No. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, Dec 07, 2011 at 04:41:17PM +0200, Avi Kivity wrote: On 12/05/2011 10:18 PM, Eric B Munson wrote: Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Long scheduling delays should be considered hangups from the guest perspective. About SIGSTOP, that is a corner case. Unsure if its even properly supported by QEMU. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/14/2011 02:16 PM, Marcelo Tosatti wrote: Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Long scheduling delays should be considered hangups from the guest perspective. Why? To the guest it looks like slow hardware, but it will interpret it as a softlockup. About SIGSTOP, that is a corner case. Unsure if its even properly supported by QEMU. It works from my experience. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/14/2011 08:39 AM, Avi Kivity wrote: On 12/14/2011 02:16 PM, Marcelo Tosatti wrote: Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Long scheduling delays should be considered hangups from the guest perspective. Why? To the guest it looks like slow hardware, but it will interpret it as a softlockup. About SIGSTOP, that is a corner case. Unsure if its even properly supported by QEMU. It works from my experience. We don't adjust vm_clock's offset on SIGSTOP/SIGCONT so while it may appear to work okay, QEMU isn't exhibiting the behavior it is supposed to. Regards, Anthony Liguori -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, 14 Dec 2011, Avi Kivity wrote: On 12/14/2011 02:16 PM, Marcelo Tosatti wrote: Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Long scheduling delays should be considered hangups from the guest perspective. Why? To the guest it looks like slow hardware, but it will interpret it as a softlockup. About SIGSTOP, that is a corner case. Unsure if its even properly supported by QEMU. It works from my experience. -- error compiling committee.c: too many arguments to function And if the soft lockup warning is a concern here, we can have a SIGCONT handler set the flag the same way qemu stop/cont does. signature.asc Description: Digital signature
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/14/2011 04:49 PM, Anthony Liguori wrote: About SIGSTOP, that is a corner case. Unsure if its even properly supported by QEMU. It works from my experience. We don't adjust vm_clock's offset on SIGSTOP/SIGCONT so while it may appear to work okay, QEMU isn't exhibiting the behavior it is supposed to. Should we trap and handle SIGCONT? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/08/2011 05:19 PM, Eric B Munson wrote: error compiling committee.c: too many arguments to function My concern for preempt notifiers is masking real soft lockup warnings. If the flag is set every time the vm is preempted, it becomes more likely that we will mask real warnings. The ioctl was choosen because it sets the flag only when the guest is being paused deliberately. AFAIK, SIGSTOP is not a supported way to stop a qemu vm so a soft lockup This is a kvm patch, not a qemu patch. In general we try to address general requirements, not just those that are specific to one userspace, even if it is near to our hearts. warning would be working as designed there. If that isn't the case, or if it ever changes, we could always install a signal handler for SIGCONT that set the flag before resuming the vm. Scheduling delays are also beyond the scope of this problem and I see the soft lockup warning as appropriate in that case. I don't think you should see a guest softlockup if the host is overloaded. Nor should you see it due to a long live migration pause, or STOP/CONT. You should see a guest softlockup if it is spinning due to a guest bug, and not for any other reason. I think we need a mix of the ioctl (for STOP/CONT and voluntary pauses, like live migration or qemu stop/cont) and preempt notifiers (together with a check for TASK_RUNNING/TASK_UNINTERRUPTIBLE, for scheduling or swap delays) -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, 14 Dec 2011, Avi Kivity wrote: On 12/08/2011 05:19 PM, Eric B Munson wrote: error compiling committee.c: too many arguments to function My concern for preempt notifiers is masking real soft lockup warnings. If the flag is set every time the vm is preempted, it becomes more likely that we will mask real warnings. The ioctl was choosen because it sets the flag only when the guest is being paused deliberately. AFAIK, SIGSTOP is not a supported way to stop a qemu vm so a soft lockup This is a kvm patch, not a qemu patch. In general we try to address general requirements, not just those that are specific to one userspace, even if it is near to our hearts. warning would be working as designed there. If that isn't the case, or if it ever changes, we could always install a signal handler for SIGCONT that set the flag before resuming the vm. Scheduling delays are also beyond the scope of this problem and I see the soft lockup warning as appropriate in that case. I don't think you should see a guest softlockup if the host is overloaded. Nor should you see it due to a long live migration pause, or STOP/CONT. You should see a guest softlockup if it is spinning due to a guest bug, and not for any other reason. I think we need a mix of the ioctl (for STOP/CONT and voluntary pauses, like live migration or qemu stop/cont) and preempt notifiers (together with a check for TASK_RUNNING/TASK_UNINTERRUPTIBLE, for scheduling or swap delays) -- error compiling committee.c: too many arguments to function Do you want the preemption notifier work in before you will consider merging this set, or can that be a follow on? Eric signature.asc Description: Digital signature
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, Dec 14, 2011 at 04:39:56PM +0200, Avi Kivity wrote: On 12/14/2011 02:16 PM, Marcelo Tosatti wrote: Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Long scheduling delays should be considered hangups from the guest perspective. Why? To the guest it looks like slow hardware, but it will interpret it as a softlockup. Slow enough that progress of the watchdog thread is unable to keep up with timer interrupt processing. This is considered a hang and should be reported. About SIGSTOP, that is a corner case. Unsure if its even properly supported by QEMU. It works from my experience. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/07/2011 04:41 PM, Avi Kivity wrote: On 12/05/2011 10:18 PM, Eric B Munson wrote: Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Guests need to run NTP regardless, not only the virtualization layer add some skew, the physical world is not that perfect. btw: traditional NTP client won't sync the time automatically if the diff is 0.5%. Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? Isn't it solved by steal time? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On (Mon) 05 Dec 2011 [15:18:59], Eric B Munson wrote: When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. Guest S4 would need similar treatment, and I think the code in the two approaches can be shared. Just something to consider. Amit -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On Wed, 07 Dec 2011, Avi Kivity wrote: On 12/05/2011 10:18 PM, Eric B Munson wrote: Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? -- error compiling committee.c: too many arguments to function My concern for preempt notifiers is masking real soft lockup warnings. If the flag is set every time the vm is preempted, it becomes more likely that we will mask real warnings. The ioctl was choosen because it sets the flag only when the guest is being paused deliberately. AFAIK, SIGSTOP is not a supported way to stop a qemu vm so a soft lockup warning would be working as designed there. If that isn't the case, or if it ever changes, we could always install a signal handler for SIGCONT that set the flag before resuming the vm. Scheduling delays are also beyond the scope of this problem and I see the soft lockup warning as appropriate in that case. Eric signature.asc Description: Digital signature
Re: [PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
On 12/05/2011 10:18 PM, Eric B Munson wrote: Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Having this controlled from userspace means it doesn't work for SIGSTOP or for long scheduling delays. What about doing this automatically based on preempt notifiers? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/5 V5] Avoid soft lockup message when KVM is stopped by host
Changes from V4: Rename KVM_GUEST_PAUSED to KVMCLOCK_GUEST_PAUSED Add description of KVMCLOCK_GUEST_PAUSED ioctl to api.txt Changes from V3: Include CC's on patch 3 Drop clear flag ioctl and have the watchdog clear the flag when it is reset Changes from V2: A new kvm functions defined in kvm_para.h, the only change to pvclock is the initial flag definition Changes from V1: (Thanks Marcelo) Host code has all been moved to arch/x86/kvm/x86.c KVM_PAUSE_GUEST was renamed to KVM_GUEST_PAUSED When a guest kernel is stopped by the host hypervisor it can look like a soft lockup to the guest kernel. This false warning can mask later soft lockup warnings which may be real. This patch series adds a method for a host hypervisor to communicate to a guest kernel that it is being stopped. The final patch in the series has the watchdog check this flag when it goes to issue a soft lockup warning and skip the warning if the guest knows it was stopped. It was attempted to solve this in Qemu, but the side effects of saving and restoring the clock and tsc for each vcpu put the wall clock of the guest behind by the amount of time of the pause. This forces a guest to have ntp running in order to keep the wall clock accurate. Cc: mi...@redhat.com Cc: h...@zytor.com Cc: a...@arndb.de Cc: ry...@linux.vnet.ibm.com Cc: aligu...@us.ibm.com Cc: mtosa...@redhat.com Cc: jeremy.fitzhardi...@citrix.com Cc: levinsasha...@gmail.com Cc: Jan Kiszka jan.kis...@siemens.com Cc: kvm@vger.kernel.org Cc: linux-a...@vger.kernel.org Cc: x...@kernel.org Cc: linux-ker...@vger.kernel.org Eric B Munson (5): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add ioctl for KVMCLOCK_GUEST_STOPPED Add generic stubs for kvm stop check functions Add check for suspended vm in softlockup detector Documentation/virtual/kvm/api.txt | 12 arch/x86/include/asm/kvm_host.h|2 ++ arch/x86/include/asm/kvm_para.h|1 + arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/kernel/kvmclock.c | 21 + arch/x86/kvm/x86.c | 20 include/asm-generic/kvm_para.h | 14 ++ include/linux/kvm.h|2 ++ kernel/watchdog.c | 12 9 files changed, 85 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/kvm_para.h -- 1.7.5.4 -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
