Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-12 Thread Rik van Riel
On 01/07/2011 12:29 AM, Mike Galbraith wrote: +#ifdef CONFIG_SMP + /* +* If this yield is important enough to want to preempt instead +* of only dropping a -next hint, we're alone, and the target +* is not alone, pull the target to this cpu. +* +*

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-12 Thread Mike Galbraith
On Wed, 2011-01-12 at 22:02 -0500, Rik van Riel wrote: Cgroups only makes the matter worse - libvirt places each KVM guest into its own cgroup, so a VCPU will generally always be alone on its own per-cgroup, per-cpu runqueue! That can lead to pulling a VCPU onto our local CPU because we

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-12 Thread Rik van Riel
On 01/12/2011 10:26 PM, Mike Galbraith wrote: On Wed, 2011-01-12 at 22:02 -0500, Rik van Riel wrote: Cgroups only makes the matter worse - libvirt places each KVM guest into its own cgroup, so a VCPU will generally always be alone on its own per-cgroup, per-cpu runqueue! That can lead to

Re: [Fwd: Re: [RFC -v3 PATCH 2/3] sched: add yield_to function]

2011-01-06 Thread Hillf Danton
...@redhat.com Cc: Hillf Dantondhi...@gmail.com,kvm@vger.kernel.org, linux-ker...@vger.kernel.org, Avi Kivitia...@redhat.com, Srivatsa Vaddagiriva...@linux.vnet.ibm.com, Mike Galbraithefa...@gmx.de, Chris Wrightchr...@sous-sol.org Subject: Re: [RFC -v3 PATCH 2/3] sched: add yield_to function Date

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-06 Thread Hillf Danton
On Thu, Jan 6, 2011 at 12:57 AM, Mike Galbraith efa...@gmx.de wrote: sched: Add yield_to(task, preempt) functionality. Currently only implemented for fair class tasks. Add a yield_to_task method() to the fair scheduling class. allowing the caller of yield_to() to accelerate another thread in

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-06 Thread Mike Galbraith
On Wed, 2011-01-05 at 18:04 +0100, Peter Zijlstra wrote: On Wed, 2011-01-05 at 17:57 +0100, Mike Galbraith wrote: + p_cfs_rq = cfs_rq_of(pse); + local = 1; + } +#endif + + /* Tell the scheduler that we'd really like pse to run next. */ +

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Avi Kivity
On 01/05/2011 04:39 AM, KOSAKI Motohiro wrote: On 01/04/2011 08:14 AM, KOSAKI Motohiro wrote: Also, If pthread_cond_signal() call sys_yield_to imlicitly, we can avoid almost Nehalem (and other P2P cache arch) lock unfairness problem. (probaby creating

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Avi Kivity
On 01/05/2011 10:40 AM, KOSAKI Motohiro wrote: On 01/05/2011 04:39 AM, KOSAKI Motohiro wrote: On 01/04/2011 08:14 AM, KOSAKI Motohiro wrote: Also, If pthread_cond_signal() call sys_yield_to imlicitly, we can avoid almost Nehalem (and other P2P cache arch) lock

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread KOSAKI Motohiro
On 01/05/2011 10:40 AM, KOSAKI Motohiro wrote: On 01/05/2011 04:39 AM, KOSAKI Motohiro wrote: On 01/04/2011 08:14 AM, KOSAKI Motohiro wrote: Also, If pthread_cond_signal() call sys_yield_to imlicitly, we can avoid almost Nehalem (and other P2P cache arch)

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Avi Kivity
On 01/05/2011 11:30 AM, KOSAKI Motohiro wrote: On 01/05/2011 10:40 AM, KOSAKI Motohiro wrote: On 01/05/2011 04:39 AM, KOSAKI Motohiro wrote: On 01/04/2011 08:14 AM, KOSAKI Motohiro wrote: Also, If pthread_cond_signal() call sys_yield_to imlicitly, we can

Re: [Fwd: Re: [RFC -v3 PATCH 2/3] sched: add yield_to function]

2011-01-05 Thread Peter Zijlstra
...@vger.kernel.org, Avi Kivitia...@redhat.com, Srivatsa Vaddagiriva...@linux.vnet.ibm.com, Mike Galbraithefa...@gmx.de, Chris Wrightchr...@sous-sol.org Subject: Re: [RFC -v3 PATCH 2/3] sched: add yield_to function Date: Tue, 04 Jan 2011 19:05:54 +0100 RT guests don't make sense, there's

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Peter Zijlstra
On Wed, 2011-01-05 at 11:39 +0900, KOSAKI Motohiro wrote: After calling pthread_cond_signal(), T1 which cond_signal caller and T2 which waked start to GIL grab race. But usually T1 is always win because lock variable is in T1's cpu cache. Why kernel and userland have so much different result?

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Peter Zijlstra
On Wed, 2011-01-05 at 17:40 +0900, KOSAKI Motohiro wrote: If you are interesting GIL mess and issue, please feel free to ask more. I suggest looking into an explicit round-robin scheme, where each thread adds itself to a queue and an unlock wakes up the first waiter. I'm sure you

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Mike Galbraith
It's Rik's patch to do with whatever he wants (I donated suggestion;), but I took the liberty of cleaning it up a bit, see below. It's essentially the same, but fixes a bug where the caller may have targeted a task in it's thread group etc, but when preemption decision time came, may have

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Peter Zijlstra
On Wed, 2011-01-05 at 17:57 +0100, Mike Galbraith wrote: + p_cfs_rq = cfs_rq_of(pse); + local = 1; + } +#endif + + /* Tell the scheduler that we'd really like pse to run next. */ + p_cfs_rq-next = pse; + + /* We know whether we want to

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Avi Kivity
On 01/04/2011 08:04 PM, Peter Zijlstra wrote: This definitely wants to be EXPORT_SYMBOL_GPL() and if it were possible I'd make it so only kvm.o could use it. It really sucks that kvm is a module. Why does it suck? I mean apart from the virtualization is crap song. -- error compiling

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Avi Kivity
On 01/05/2011 07:15 PM, Peter Zijlstra wrote: On Wed, 2011-01-05 at 19:10 +0200, Avi Kivity wrote: On 01/04/2011 08:04 PM, Peter Zijlstra wrote: This definitely wants to be EXPORT_SYMBOL_GPL() and if it were possible I'd make it so only kvm.o could use it. It really sucks that kvm is a

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Mike Galbraith
On Wed, 2011-01-05 at 18:04 +0100, Peter Zijlstra wrote: On Wed, 2011-01-05 at 17:57 +0100, Mike Galbraith wrote: + p_cfs_rq = cfs_rq_of(pse); + local = 1; + } +#endif + + /* Tell the scheduler that we'd really like pse to run next. */ +

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Peter Zijlstra
On Wed, 2011-01-05 at 19:19 +0200, Avi Kivity wrote: On 01/05/2011 07:15 PM, Peter Zijlstra wrote: On Wed, 2011-01-05 at 19:10 +0200, Avi Kivity wrote: On 01/04/2011 08:04 PM, Peter Zijlstra wrote: This definitely wants to be EXPORT_SYMBOL_GPL() and if it were possible I'd

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Avi Kivity
On 01/05/2011 07:28 PM, Peter Zijlstra wrote: On Wed, 2011-01-05 at 19:19 +0200, Avi Kivity wrote: On 01/05/2011 07:15 PM, Peter Zijlstra wrote: On Wed, 2011-01-05 at 19:10 +0200, Avi Kivity wrote: On 01/04/2011 08:04 PM, Peter Zijlstra wrote: This definitely wants to be

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Peter Zijlstra
On Wed, 2011-01-05 at 19:35 +0200, Avi Kivity wrote: Tejun, why did you end up not using preempt_notifiers in cmwq? Because I told him to use explicit function calls because that keeps the code easier to read. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-05 Thread Peter Zijlstra
On Wed, 2011-01-05 at 19:10 +0200, Avi Kivity wrote: On 01/04/2011 08:04 PM, Peter Zijlstra wrote: This definitely wants to be EXPORT_SYMBOL_GPL() and if it were possible I'd make it so only kvm.o could use it. It really sucks that kvm is a module. Why does it suck? I mean apart from

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Avi Kivity
On 01/04/2011 08:14 AM, KOSAKI Motohiro wrote: Also, If pthread_cond_signal() call sys_yield_to imlicitly, we can avoid almost Nehalem (and other P2P cache arch) lock unfairness problem. (probaby creating pthread_condattr_setautoyield_np or similar knob is good one) Often, the thread calling

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Hillf Danton
On Tue, Jan 4, 2011 at 5:29 AM, Rik van Riel r...@redhat.com wrote: From: Mike Galbraith efa...@gmx.de Add a yield_to function to the scheduler code, allowing us to give enough of our timeslice to another thread to allow it to run and release whatever resource we need it to release. We may

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Hillf Danton
On Tue, Jan 4, 2011 at 5:29 AM, Rik van Riel r...@redhat.com wrote: From: Mike Galbraith efa...@gmx.de Add a yield_to function to the scheduler code, allowing us to give enough of our timeslice to another thread to allow it to run and release whatever resource we need it to release. We may

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Rik van Riel
On 01/04/2011 11:41 AM, Hillf Danton wrote: /* !curr-sched_class-yield_to_task ||*/ + curr-sched_class != p-sched_class) { + goto out; + } + /* * ask scheduler to compute the next for successfully

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Hillf Danton
On Wed, Jan 5, 2011 at 12:44 AM, Rik van Riel r...@redhat.com wrote: On 01/04/2011 11:41 AM, Hillf Danton wrote: /*                       !curr-sched_class-yield_to_task ||        */ +                       curr-sched_class != p-sched_class) { +               goto out; +       } +        

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Rik van Riel
On 01/04/2011 11:51 AM, Hillf Danton wrote: Wouldn't that break for FIFO and RR tasks? There's a reason all the scheduler folks wanted a per-class yield_to_task function :) Where is the yield_to callback in the patch for RT schedule class? If @p is RT, what could you do? If the user

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Hillf Danton
On Wed, Jan 5, 2011 at 12:54 AM, Rik van Riel r...@redhat.com wrote: On 01/04/2011 11:51 AM, Hillf Danton wrote: Wouldn't that break for FIFO and RR tasks? There's a reason all the scheduler folks wanted a per-class yield_to_task function :) Where is the yield_to callback in the patch for

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Peter Zijlstra
On Wed, 2011-01-05 at 00:51 +0800, Hillf Danton wrote: Where is the yield_to callback in the patch for RT schedule class? If @p is RT, what could you do? RT guests are a pipe dream, you first need to get the hypervisor (kvm in this case) to be RT, which it isn't. Then you either need to very

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Hillf Danton
On Wed, Jan 5, 2011 at 1:08 AM, Peter Zijlstra a.p.zijls...@chello.nl wrote: On Wed, 2011-01-05 at 00:51 +0800, Hillf Danton wrote: Where is the yield_to callback in the patch for RT schedule class? If @p is RT, what could you do? RT guests are a pipe dream, you first need to get the

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Peter Zijlstra
On Tue, 2011-01-04 at 15:14 +0900, KOSAKI Motohiro wrote: From: Mike Galbraith efa...@gmx.de Add a yield_to function to the scheduler code, allowing us to give enough of our timeslice to another thread to allow it to run and release whatever resource we need it to release. We may

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Peter Zijlstra
On Wed, 2011-01-05 at 01:12 +0800, Hillf Danton wrote: On Wed, Jan 5, 2011 at 1:08 AM, Peter Zijlstra a.p.zijls...@chello.nl wrote: On Wed, 2011-01-05 at 00:51 +0800, Hillf Danton wrote: Where is the yield_to callback in the patch for RT schedule class? If @p is RT, what could you do?

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Rik van Riel
On 01/04/2011 12:08 PM, Peter Zijlstra wrote: On Wed, 2011-01-05 at 00:51 +0800, Hillf Danton wrote: Where is the yield_to callback in the patch for RT schedule class? If @p is RT, what could you do? RT guests are a pipe dream, you first need to get the hypervisor (kvm in this case) to be RT,

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Peter Zijlstra
On Mon, 2011-01-03 at 16:29 -0500, Rik van Riel wrote: From: Mike Galbraith efa...@gmx.de Add a yield_to function to the scheduler code, allowing us to give enough of our timeslice to another thread to allow it to run and release whatever resource we need it to release. We may want to use

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread Mike Galbraith
On Tue, 2011-01-04 at 19:04 +0100, Peter Zijlstra wrote: + p_cfs_rq = cfs_rq_of(pse); + yield = 1; + } +#endif + + if (yield) + clear_buddies(cfs_rq, se); + else if (preempt) + clear_buddies(p_cfs_rq, curr); + + /* Tell the

Re: [Fwd: Re: [RFC -v3 PATCH 2/3] sched: add yield_to function]

2011-01-04 Thread Tommaso Cucinotta
...@linux.vnet.ibm.com, Mike Galbraithefa...@gmx.de, Chris Wrightchr...@sous-sol.org Subject: Re: [RFC -v3 PATCH 2/3] sched: add yield_to function Date: Tue, 04 Jan 2011 19:05:54 +0100 RT guests don't make sense, there's nowhere near enough infrastructure for that to work well. I'd argue that KVM

Re: [Fwd: Re: [RFC -v3 PATCH 2/3] sched: add yield_to function]

2011-01-04 Thread Tommaso Cucinotta
Il 05/01/2011 00:38, Tommaso Cucinotta ha scritto: Of course, don't misunderstand me: this is a necessary condition for a stable performance of KVM VMs, I'm not saying it is sufficient for ... it. Please, comment on this (reply to all, please, I'm not following LKML). Thanks, Tommaso --

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread KOSAKI Motohiro
On 01/04/2011 08:14 AM, KOSAKI Motohiro wrote: Also, If pthread_cond_signal() call sys_yield_to imlicitly, we can avoid almost Nehalem (and other P2P cache arch) lock unfairness problem. (probaby creating pthread_condattr_setautoyield_np or similar knob is good one) Often, the thread

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-04 Thread KOSAKI Motohiro
NAK NAK NAK, yield_to is utter crap, and the only reason kvm 'needs' it is because its wants to be utter crap (run unmodified guests). There is plenty of sane serialization primitives for userspace, fix your locking mess instead of pushing crap. The only reason I'm maybe half-way

[RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-03 Thread Rik van Riel
From: Mike Galbraith efa...@gmx.de Add a yield_to function to the scheduler code, allowing us to give enough of our timeslice to another thread to allow it to run and release whatever resource we need it to release. We may want to use this to provide a sys_yield_to system call one day.

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-03 Thread Mike Galbraith
On Mon, 2011-01-03 at 16:29 -0500, Rik van Riel wrote: From: Mike Galbraith efa...@gmx.de Add a yield_to function to the scheduler code, allowing us to give enough of our timeslice to another thread to allow it to run and release whatever resource we need it to release. We may want to use

Re: [RFC -v3 PATCH 2/3] sched: add yield_to function

2011-01-03 Thread KOSAKI Motohiro
From: Mike Galbraith efa...@gmx.de Add a yield_to function to the scheduler code, allowing us to give enough of our timeslice to another thread to allow it to run and release whatever resource we need it to release. We may want to use this to provide a sys_yield_to system call one day.