AW: KVM and Prempt?
-Ursprüngliche Nachricht- Von: Kiszka, Jan Gesendet: Tuesday, October 23, 2007 10:35 AM An: Sven-Thorsten Dietrich Cc: Back, Michael (ext); linux-rt-users@vger.kernel.org Betreff: Re: KVM and Prempt? Sven-Thorsten Dietrich wrote: On Mon, 2007-10-22 at 09:01 +0200, Back, Michael (ext) wrote: Hallo, I tried to run Windows XP with KVM on Linux 2.6.31.1 on a You mean .21.1 ? Classic typo I interestingly also did several times the last week. :) AMD Opteron and on a Intel Xeon, on both it works fine! After this test I patch the kernel with the current prempt-patch and on both it doesn't works! Did you try against 2.6.23-rt1. kvm in -rt1 is not usable. It's too old, lacking PREEMPT_NOTIFIER support, thus quickly triggering lockdep. Thanks, the kvm-devel list told me the same and it works !!! Michael If you must stay on .21, you might have some other issues with the AMD and NUMA. At the very least, you will need to apply the attached patch from git somehow, although this patch is against a new scheduler post 2.6.22, so good luck :) --snip-- Those patches are already mainline... :- What you rather need are latest kvm patches, or - if building the kvm distribution out of tree - a patch to enabled CONFIG_PREEMPT_NOTIFIERS unconditionally: --- linux-2.6.23.1-rt/kernel/Kconfig.preempt.orig +++ linux-2.6.23.1-rt/kernel/Kconfig.preempt @@ -136,6 +136,7 @@ config PREEMPT_NOTIFIERS bool + default y config PREEMPT_BKL bool Still, I'm seeing oopses here (more precisely, lock validator complaints), but I need to re-test, better using kvm from git instead of kvm-48. Beyond this, I'm struggling to understand 300-400 us vm-exit latencies (over Intel VMX), which appear to be independent of the underlying system. See kvm-devel. Such latencies would limit the RT usability of kvm - unless you spent dedicated CPUs. Jan -- Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux - To unsubscribe from this list: send the line unsubscribe linux-rt-users in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM and Prempt?
Sven-Thorsten Dietrich wrote: On Mon, 2007-10-22 at 09:01 +0200, Back, Michael (ext) wrote: Hallo, I tried to run Windows XP with KVM on Linux 2.6.31.1 on a You mean .21.1 ? Classic typo I interestingly also did several times the last week. :) AMD Opteron and on a Intel Xeon, on both it works fine! After this test I patch the kernel with the current prempt-patch and on both it doesn't works! Did you try against 2.6.23-rt1. kvm in -rt1 is not usable. It's too old, lacking PREEMPT_NOTIFIER support, thus quickly triggering lockdep. If you must stay on .21, you might have some other issues with the AMD and NUMA. At the very least, you will need to apply the attached patch from git somehow, although this patch is against a new scheduler post 2.6.22, so good luck :) --snip-- Those patches are already mainline... :- What you rather need are latest kvm patches, or - if building the kvm distribution out of tree - a patch to enabled CONFIG_PREEMPT_NOTIFIERS unconditionally: --- linux-2.6.23.1-rt/kernel/Kconfig.preempt.orig +++ linux-2.6.23.1-rt/kernel/Kconfig.preempt @@ -136,6 +136,7 @@ config PREEMPT_NOTIFIERS bool + default y config PREEMPT_BKL bool Still, I'm seeing oopses here (more precisely, lock validator complaints), but I need to re-test, better using kvm from git instead of kvm-48. Beyond this, I'm struggling to understand 300-400 us vm-exit latencies (over Intel VMX), which appear to be independent of the underlying system. See kvm-devel. Such latencies would limit the RT usability of kvm - unless you spent dedicated CPUs. Jan -- Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux - To unsubscribe from this list: send the line unsubscribe linux-rt-users in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM and Prempt?
On Tue, 2007-10-23 at 10:35 +0200, Jan Kiszka wrote: Sven-Thorsten Dietrich wrote: On Mon, 2007-10-22 at 09:01 +0200, Back, Michael (ext) wrote: Hallo, I tried to run Windows XP with KVM on Linux 2.6.31.1 on a You mean .21.1 ? Classic typo I interestingly also did several times the last week. :) AMD Opteron and on a Intel Xeon, on both it works fine! After this test I patch the kernel with the current prempt-patch and on both it doesn't works! Did you try against 2.6.23-rt1. kvm in -rt1 is not usable. It's too old, lacking PREEMPT_NOTIFIER support, thus quickly triggering lockdep. If you must stay on .21, you might have some other issues with the AMD and NUMA. At the very least, you will need to apply the attached patch from git somehow, although this patch is against a new scheduler post 2.6.22, so good luck :) --snip-- Those patches are already mainline... :- What you rather need are latest kvm patches, or - if building the kvm distribution out of tree - a patch to enabled CONFIG_PREEMPT_NOTIFIERS unconditionally: --- linux-2.6.23.1-rt/kernel/Kconfig.preempt.orig +++ linux-2.6.23.1-rt/kernel/Kconfig.preempt @@ -136,6 +136,7 @@ config PREEMPT_NOTIFIERS bool + default y config PREEMPT_BKL bool Still, I'm seeing oopses here (more precisely, lock validator complaints), but I need to re-test, better using kvm from git instead of kvm-48. Beyond this, I'm struggling to understand 300-400 us vm-exit latencies (over Intel VMX), which appear to be independent of the underlying system. See kvm-devel. Such latencies would limit the RT usability of kvm - unless you spent dedicated CPUs. Some work is still left to be done in this area. Until then you will see problems like you are describing. At one point I had a patch series that allowed KVM to actually work in -rt without crashes, and with decent latencies (both host, and guest). Some point soon I will revive the series and port it to the latest kvm.git. HTH Regards, -Greg signature.asc Description: This is a digitally signed message part
KVM and Prempt?
Hallo, I tried to run Windows XP with KVM on Linux 2.6.31.1 on a AMD Opteron and on a Intel Xeon, on both it works fine! After this test I patch the kernel with the current prempt-patch and on both it doesn't works! - After a very short time - I could see the windows startup screen - the complied system froze! Has you ever tried to do the same and it works? Or will KVM with Windows on a prempt kernel - never work? - maybe work in the future? - should now work but this .. and this ... should be done and consider? With best regards, Michael - To unsubscribe from this list: send the line unsubscribe linux-rt-users in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM and Prempt?
On Mon, 2007-10-22 at 09:01 +0200, Back, Michael (ext) wrote: Hallo, I tried to run Windows XP with KVM on Linux 2.6.31.1 on a You mean .21.1 ? AMD Opteron and on a Intel Xeon, on both it works fine! After this test I patch the kernel with the current prempt-patch and on both it doesn't works! Did you try against 2.6.23-rt1. If you must stay on .21, you might have some other issues with the AMD and NUMA. At the very least, you will need to apply the attached patch from git somehow, although this patch is against a new scheduler post 2.6.22, so good luck :) Sven - After a very short time - I could see the windows startup screen - the complied system froze! Has you ever tried to do the same and it works? Or will KVM with Windows on a prempt kernel - never work? - maybe work in the future? - should now work but this .. and this ... should be done and consider? With best regards, Michael - To unsubscribe from this list: send the line unsubscribe linux-rt-users in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html diff-tree e107be36efb2a233833e8c9899039a370e4b2318 (from b47e8608a08766ef8121cd747d3aaf6c3dc22649) Author: Avi Kivity [EMAIL PROTECTED] Date: Thu Jul 26 13:40:43 2007 +0200 [PATCH] sched: arch preempt notifier mechanism This adds a general mechanism whereby a task can request the scheduler to notify it whenever it is preempted or scheduled back in. This allows the task to swap any special-purpose registers like the fpu or Intel's VT registers. Signed-off-by: Avi Kivity [EMAIL PROTECTED] [ [EMAIL PROTECTED]: fixes, cleanups ] Signed-off-by: Ingo Molnar [EMAIL PROTECTED] diff --git a/include/linux/preempt.h b/include/linux/preempt.h index d0926d6..484988e 100644 --- a/include/linux/preempt.h +++ b/include/linux/preempt.h @@ -8,6 +8,7 @@ #include linux/thread_info.h #include linux/linkage.h +#include linux/list.h #ifdef CONFIG_DEBUG_PREEMPT extern void fastcall add_preempt_count(int val); @@ -60,4 +61,47 @@ do { \ #endif +#ifdef CONFIG_PREEMPT_NOTIFIERS + +struct preempt_notifier; + +/** + * preempt_ops - notifiers called when a task is preempted and rescheduled + * @sched_in: we're about to be rescheduled: + *notifier: struct preempt_notifier for the task being scheduled + *cpu: cpu we're scheduled on + * @sched_out: we've just been preempted + *notifier: struct preempt_notifier for the task being preempted + *next: the task that's kicking us out + */ +struct preempt_ops { + void (*sched_in)(struct preempt_notifier *notifier, int cpu); + void (*sched_out)(struct preempt_notifier *notifier, + struct task_struct *next); +}; + +/** + * preempt_notifier - key for installing preemption notifiers + * @link: internal use + * @ops: defines the notifier functions to be called + * + * Usually used in conjunction with container_of(). + */ +struct preempt_notifier { + struct hlist_node link; + struct preempt_ops *ops; +}; + +void preempt_notifier_register(struct preempt_notifier *notifier); +void preempt_notifier_unregister(struct preempt_notifier *notifier); + +static inline void preempt_notifier_init(struct preempt_notifier *notifier, +struct preempt_ops *ops) +{ + INIT_HLIST_NODE(notifier-link); + notifier-ops = ops; +} + +#endif + #endif /* __LINUX_PREEMPT_H */ diff --git a/include/linux/sched.h b/include/linux/sched.h index 7c61b50..7a4de87 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -935,6 +935,11 @@ struct task_struct { struct sched_class *sched_class; struct sched_entity se; +#ifdef CONFIG_PREEMPT_NOTIFIERS + /* list of struct preempt_notifier: */ + struct hlist_head preempt_notifiers; +#endif + unsigned short ioprio; #ifdef CONFIG_BLK_DEV_IO_TRACE unsigned int btrace_seq; diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt index c64ce9c..6b06663 100644 --- a/kernel/Kconfig.preempt +++ b/kernel/Kconfig.preempt @@ -63,3 +63,6 @@ config PREEMPT_BKL Say Y here if you are building a kernel for a desktop system. Say N if you are unsure. +config PREEMPT_NOTIFIERS + bool + diff --git a/kernel/sched.c b/kernel/sched.c index 93cf241..e901aa5 100644 --- a/kernel/sched.c +++ b/kernel/sched.c @@ -1592,6 +1592,10 @@ static void __sched_fork(struct task_str INIT_LIST_HEAD(p-run_list); p-se.on_rq = 0; +#ifdef CONFIG_PREEMPT_NOTIFIERS + INIT_HLIST_HEAD(p-preempt_notifiers); +#endif + /* * We mark the process as running here, but have not actually * inserted it onto the runqueue yet. This guarantees that @@ -1673,6 +1677,63 @@ void fastcall wake_up_new_task(struct ta task_rq_unlock(rq, flags); } +#ifdef CONFIG_PREEMPT_NOTIFIERS + +/** + * preempt_notifier_register - tell me when current is