Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, Nov 29, 2017 at 07:15:21PM +0100, Mike Galbraith wrote: > On Wed, 2017-11-29 at 11:41 +0100, Uladzislau Rezki wrote: > > On Tue, Nov 28, 2017 at 11:49:11AM +0100, Mike Galbraith wrote: > > > On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > > > > On Fri, Nov 24, 2017 at

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, Nov 29, 2017 at 07:15:21PM +0100, Mike Galbraith wrote: > On Wed, 2017-11-29 at 11:41 +0100, Uladzislau Rezki wrote: > > On Tue, Nov 28, 2017 at 11:49:11AM +0100, Mike Galbraith wrote: > > > On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > > > > On Fri, Nov 24, 2017 at

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-29 at 11:41 +0100, Uladzislau Rezki wrote: > On Tue, Nov 28, 2017 at 11:49:11AM +0100, Mike Galbraith wrote: > > On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > > > On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > > > > > > > My view is you're barking

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-29 at 11:41 +0100, Uladzislau Rezki wrote: > On Tue, Nov 28, 2017 at 11:49:11AM +0100, Mike Galbraith wrote: > > On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > > > On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > > > > > > > My view is you're barking

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, Nov 28, 2017 at 11:49:11AM +0100, Mike Galbraith wrote: > On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > > On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > > > > > My view is you're barking up the wrong tree: you're making the idle > > > data SIS is using

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, Nov 28, 2017 at 11:49:11AM +0100, Mike Galbraith wrote: > On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > > On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > > > > > My view is you're barking up the wrong tree: you're making the idle > > > data SIS is using

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > > > My view is you're barking up the wrong tree: you're making the idle > > data SIS is using more accurate, but I question the benefit.  That it > > makes an imperfect

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, 2017-11-28 at 10:34 +0100, Uladzislau Rezki wrote: > On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > > > My view is you're barking up the wrong tree: you're making the idle > > data SIS is using more accurate, but I question the benefit.  That it > > makes an imperfect

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > On Fri, 2017-11-24 at 11:26 +0100, Uladzislau Rezki wrote: > > > > I guess there is misunderstanding here. The main goal is not to cover > > pinned case, for sure. I was thinking more about below points: > > > > - Extend a

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > On Fri, 2017-11-24 at 11:26 +0100, Uladzislau Rezki wrote: > > > > I guess there is misunderstanding here. The main goal is not to cover > > pinned case, for sure. I was thinking more about below points: > > > > - Extend a

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Fri, 2017-11-24 at 19:46 +0100, Mike Galbraith wrote: > > My view is you're barking up the wrong tree: you're making the idle > data SIS is using more accurate, but I question the benefit.  That it > makes an imperfect placement decision occasionally due to raciness is > nearly meaningless

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Fri, 2017-11-24 at 19:46 +0100, Mike Galbraith wrote: > > My view is you're barking up the wrong tree: you're making the idle > data SIS is using more accurate, but I question the benefit.  That it > makes an imperfect placement decision occasionally due to raciness is > nearly meaningless

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Fri, 2017-11-24 at 11:26 +0100, Uladzislau Rezki wrote: > > I guess there is misunderstanding here. The main goal is not to cover > pinned case, for sure. I was thinking more about below points: > > - Extend a claim_wake_up logic for making an ILB/NO_HZ decision more > predictable (that is

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Fri, 2017-11-24 at 11:26 +0100, Uladzislau Rezki wrote: > > I guess there is misunderstanding here. The main goal is not to cover > pinned case, for sure. I was thinking more about below points: > > - Extend a claim_wake_up logic for making an ILB/NO_HZ decision more > predictable (that is

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, Nov 23, 2017 at 02:13:01PM +0100, Mike Galbraith wrote: > On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: > > Hello, Atish, Peter, all. > > > > I have a question about if a task's nr_cpus_allowed is 1. > > In that scenario we do not call select_task_rq. Therefore > > even

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, Nov 23, 2017 at 02:13:01PM +0100, Mike Galbraith wrote: > On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: > > Hello, Atish, Peter, all. > > > > I have a question about if a task's nr_cpus_allowed is 1. > > In that scenario we do not call select_task_rq. Therefore > > even

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On 2017/11/23 10:00 AM, Josef Bacik wrote: On Thu, Nov 23, 2017 at 02:13:01PM +0100, Mike Galbraith wrote: On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: Hello, Atish, Peter, all. I have a question about if a task's nr_cpus_allowed is 1. In that scenario we do not call

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On 2017/11/23 10:00 AM, Josef Bacik wrote: On Thu, Nov 23, 2017 at 02:13:01PM +0100, Mike Galbraith wrote: On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: Hello, Atish, Peter, all. I have a question about if a task's nr_cpus_allowed is 1. In that scenario we do not call

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, 2017-11-23 at 11:00 -0500, Josef Bacik wrote: > > And on this thanksgiving I'm thankful for Mike, and his entertaining early > morning emails. Read it again tomorrow. -Mike

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, 2017-11-23 at 11:00 -0500, Josef Bacik wrote: > > And on this thanksgiving I'm thankful for Mike, and his entertaining early > morning emails. Read it again tomorrow. -Mike

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, Nov 23, 2017 at 02:13:01PM +0100, Mike Galbraith wrote: > On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: > > Hello, Atish, Peter, all. > > > > I have a question about if a task's nr_cpus_allowed is 1. > > In that scenario we do not call select_task_rq. Therefore > > even

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, Nov 23, 2017 at 02:13:01PM +0100, Mike Galbraith wrote: > On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: > > Hello, Atish, Peter, all. > > > > I have a question about if a task's nr_cpus_allowed is 1. > > In that scenario we do not call select_task_rq. Therefore > > even

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: > Hello, Atish, Peter, all. > > I have a question about if a task's nr_cpus_allowed is 1. > In that scenario we do not call select_task_rq. Therefore > even thought a task "p" is placed on idle CPU that CPU > will not be marked as claimed

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Thu, 2017-11-23 at 11:52 +0100, Uladzislau Rezki wrote: > Hello, Atish, Peter, all. > > I have a question about if a task's nr_cpus_allowed is 1. > In that scenario we do not call select_task_rq. Therefore > even thought a task "p" is placed on idle CPU that CPU > will not be marked as claimed

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

Hello, Atish, Peter, all. I have a question about if a task's nr_cpus_allowed is 1. In that scenario we do not call select_task_rq. Therefore even thought a task "p" is placed on idle CPU that CPU will not be marked as claimed for wake-up. What do you think about adding per_cpu(claim_wakeup,

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

Hello, Atish, Peter, all. I have a question about if a task's nr_cpus_allowed is 1. In that scenario we do not call select_task_rq. Therefore even thought a task "p" is placed on idle CPU that CPU will not be marked as claimed for wake-up. What do you think about adding per_cpu(claim_wakeup,

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

Here are the results of schbench(scheduler latency benchmark) and uperf (networking benchmark). Hardware config: 20 core (40 hyperthreaded cpus) x86 box. schbench config: message threads = 2; time = 180s, worker thread = variable uperf config:ping pong test on loopback interface with message

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

Here are the results of schbench(scheduler latency benchmark) and uperf (networking benchmark). Hardware config: 20 core (40 hyperthreaded cpus) x86 box. schbench config: message threads = 2; time = 180s, worker thread = variable uperf config:ping pong test on loopback interface with message

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

Hi Peter, On Tue, Oct 31, 2017 at 1:20 AM, Peter Zijlstra wrote: > On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: >> Currently, multiple tasks can wakeup on same cpu from >> select_idle_sibiling() path in case they wakeup simulatenously >> and last ran on the

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

Hi Peter, On Tue, Oct 31, 2017 at 1:20 AM, Peter Zijlstra wrote: > On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: >> Currently, multiple tasks can wakeup on same cpu from >> select_idle_sibiling() path in case they wakeup simulatenously >> and last ran on the same llc. This happens

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-01 at 11:36 -0500, Atish Patra wrote: > > Any other benchmark suggestion that may benefit from this fix ? Nope.  I'll be kinda surprised if you find anything where you don't have to squint to see a dinky signal modulating white noise :) -Mike

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-01 at 11:36 -0500, Atish Patra wrote: > > Any other benchmark suggestion that may benefit from this fix ? Nope.  I'll be kinda surprised if you find anything where you don't have to squint to see a dinky signal modulating white noise :) -Mike

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On 11/01/2017 02:18 AM, Mike Galbraith wrote: On Wed, 2017-11-01 at 07:54 +0100, Mike Galbraith wrote: On Wed, 2017-11-01 at 01:08 -0500, Atish Patra wrote: Do you have the schbench configuration somewhere that I can test? I tried various configurations but did not see any improvement or

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On 11/01/2017 02:18 AM, Mike Galbraith wrote: On Wed, 2017-11-01 at 07:54 +0100, Mike Galbraith wrote: On Wed, 2017-11-01 at 01:08 -0500, Atish Patra wrote: Do you have the schbench configuration somewhere that I can test? I tried various configurations but did not see any improvement or

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-01 at 07:54 +0100, Mike Galbraith wrote: > On Wed, 2017-11-01 at 01:08 -0500, Atish Patra wrote: > > > Do you have the schbench configuration somewhere that I can test? I > > tried various configurations but did not > > see any improvement or regression. > > No, as noted, I

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-01 at 07:54 +0100, Mike Galbraith wrote: > On Wed, 2017-11-01 at 01:08 -0500, Atish Patra wrote: > > > Do you have the schbench configuration somewhere that I can test? I > > tried various configurations but did not > > see any improvement or regression. > > No, as noted, I

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-01 at 01:08 -0500, Atish Patra wrote: > > On 10/31/2017 03:48 AM, Mike Galbraith wrote: > > > I played with something ~similar (cmpxchg() idle cpu reservation) > I had an atomic version earlier as well. Peter's suggestion for per cpu > seems to perform slightly better than

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Wed, 2017-11-01 at 01:08 -0500, Atish Patra wrote: > > On 10/31/2017 03:48 AM, Mike Galbraith wrote: > > > I played with something ~similar (cmpxchg() idle cpu reservation) > I had an atomic version earlier as well. Peter's suggestion for per cpu > seems to perform slightly better than

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On 10/31/2017 03:48 AM, Mike Galbraith wrote: On Tue, 2017-10-31 at 09:20 +0100, Peter Zijlstra wrote: On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: Currently, multiple tasks can wakeup on same cpu from select_idle_sibiling() path in case they wakeup simulatenously and last

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On 10/31/2017 03:48 AM, Mike Galbraith wrote: On Tue, 2017-10-31 at 09:20 +0100, Peter Zijlstra wrote: On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: Currently, multiple tasks can wakeup on same cpu from select_idle_sibiling() path in case they wakeup simulatenously and last

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, 2017-10-31 at 09:20 +0100, Peter Zijlstra wrote: > On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: > > Currently, multiple tasks can wakeup on same cpu from > > select_idle_sibiling() path in case they wakeup simulatenously > > and last ran on the same llc. This happens

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, 2017-10-31 at 09:20 +0100, Peter Zijlstra wrote: > On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: > > Currently, multiple tasks can wakeup on same cpu from > > select_idle_sibiling() path in case they wakeup simulatenously > > and last ran on the same llc. This happens

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: > Currently, multiple tasks can wakeup on same cpu from > select_idle_sibiling() path in case they wakeup simulatenously > and last ran on the same llc. This happens because an idle cpu > is not updated until idle task is scheduled out.

Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote: > Currently, multiple tasks can wakeup on same cpu from > select_idle_sibiling() path in case they wakeup simulatenously > and last ran on the same llc. This happens because an idle cpu > is not updated until idle task is scheduled out.