Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread John Baldwin
On Thursday, May 08, 2014 11:43:39 pm Adrian Chadd wrote: Hi, I'd like to revisit this now. I'd like to commit this stuff as-is and then take some time to revisit the catch-all softclock from cpu0 swi. It's more complicated than it needs to be as it just assumes timeout_cpu == cpuid of

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread Adrian Chadd
On 9 May 2014 10:49, John Baldwin j...@freebsd.org wrote: On Thursday, May 08, 2014 11:43:39 pm Adrian Chadd wrote: Hi, I'd like to revisit this now. I'd like to commit this stuff as-is and then take some time to revisit the catch-all softclock from cpu0 swi. It's more complicated than it

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread Peter Grehan
How about i instead do the comprimise: * i'll pin all other swi's * default swi isn't pinned by default, but one can flip on a sysctl at boot time to pin it How's that sound? And also please a sysctl that disables any swi pinning. It is sometimes useful to change the default cpuset, for

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread Adrian Chadd
On 9 May 2014 12:50, Peter Grehan gre...@freebsd.org wrote: How about i instead do the comprimise: * i'll pin all other swi's * default swi isn't pinned by default, but one can flip on a sysctl at boot time to pin it How's that sound? And also please a sysctl that disables any swi

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread John Baldwin
On Friday, May 09, 2014 3:50:28 pm Peter Grehan wrote: How about i instead do the comprimise: * i'll pin all other swi's * default swi isn't pinned by default, but one can flip on a sysctl at boot time to pin it How's that sound? And also please a sysctl that disables any swi

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread Peter Grehan
Yup. I've just done that. http://people.freebsd.org/~adrian/norse/20140509-swi-pin-1.diff Thanks, that'll work. Which workloads are you thinking about? Maybe we could introduce some higher level description of which CPU(s) at boot time to do freebsd stuff on, and then don't start things

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread Adrian Chadd
On 9 May 2014 16:49, Peter Grehan gre...@freebsd.org wrote: Yup. I've just done that. http://people.freebsd.org/~adrian/norse/20140509-swi-pin-1.diff Thanks, that'll work. Which workloads are you thinking about? Maybe we could introduce some higher level description of which CPU(s) at

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-09 Thread Adrian Chadd
ok, I've committed it but I've left the default at don't pin. That way the existing behaviour hasn't changed and it's easy to flip on to play with. Thanks for the feedback John / Peter! -a ___ freebsd-current@freebsd.org mailing list

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-05-08 Thread Adrian Chadd
Hi, I'd like to revisit this now. I'd like to commit this stuff as-is and then take some time to revisit the catch-all softclock from cpu0 swi. It's more complicated than it needs to be as it just assumes timeout_cpu == cpuid of cpu 0. So there's no easy way to slide in a new catch-all

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-20 Thread John Baldwin
On Wednesday, February 19, 2014 4:02:54 pm John Baldwin wrote: On Wednesday, February 19, 2014 3:04:51 pm Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So if we're moving towards supporting (among others) a pcbgroup / RSS hash style work load

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-20 Thread Adrian Chadd
On 20 February 2014 11:17, John Baldwin j...@freebsd.org wrote: (A further variant of this would be to divorce cpu0's swi from the catch-all softclock and let the catch-all softclock float, but bind all the per-cpu swis) I like this idea. If something (eg per-CPU TCP timers, if it's turned

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Alexander Motin
Hi. Clock interrupt threads, same as other ones are only softly bound to specific CPUs by scheduler preferring to run them on CPUs where they are scheduled. So far that was enough to balance load, but allowed threads to migrate, if needed. Is it too flexible for some use case? -- Alexander

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Adrian Chadd
On 19 February 2014 11:40, Alexander Motin m...@freebsd.org wrote: Hi. Clock interrupt threads, same as other ones are only softly bound to specific CPUs by scheduler preferring to run them on CPUs where they are scheduled. So far that was enough to balance load, but allowed threads to

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Alexander Motin
On 19.02.2014 21:51, Adrian Chadd wrote: On 19 February 2014 11:40, Alexander Motin m...@freebsd.org wrote: Clock interrupt threads, same as other ones are only softly bound to specific CPUs by scheduler preferring to run them on CPUs where they are scheduled. So far that was enough to balance

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Adrian Chadd
On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So if we're moving towards supporting (among others) a pcbgroup / RSS hash style work load distribution across CPUs to minimise per-connection lock contention, we really don't want the scheduler to decide it can schedule

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Alfred Perlstein
On 2/19/14, 12:04 PM, Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So if we're moving towards supporting (among others) a pcbgroup / RSS hash style work load distribution across CPUs to minimise per-connection lock contention, we really don't want the

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread John Baldwin
On Wednesday, February 19, 2014 3:04:51 pm Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So if we're moving towards supporting (among others) a pcbgroup / RSS hash style work load distribution across CPUs to minimise per-connection lock contention,

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Alexander Motin
On 19.02.2014 22:04, Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So if we're moving towards supporting (among others) a pcbgroup / RSS hash style work load distribution across CPUs to minimise per-connection lock contention, we really don't want the

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Slawa Olhovchenkov
On Wed, Feb 19, 2014 at 11:04:49PM +0200, Alexander Motin wrote: On 19.02.2014 22:04, Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So if we're moving towards supporting (among others) a pcbgroup / RSS hash style work load distribution across CPUs

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Alexander Motin
On 19.02.2014 23:44, Slawa Olhovchenkov wrote: On Wed, Feb 19, 2014 at 11:04:49PM +0200, Alexander Motin wrote: On 19.02.2014 22:04, Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So if we're moving towards supporting (among others) a pcbgroup / RSS

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Adrian Chadd
On 19 February 2014 14:09, Alexander Motin m...@freebsd.org wrote: On 19.02.2014 23:44, Slawa Olhovchenkov wrote: On Wed, Feb 19, 2014 at 11:04:49PM +0200, Alexander Motin wrote: On 19.02.2014 22:04, Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote: So

Re: [rfc] bind per-cpu timeout threads to each CPU

2014-02-19 Thread Slawa Olhovchenkov
On Thu, Feb 20, 2014 at 12:09:04AM +0200, Alexander Motin wrote: On 19.02.2014 23:44, Slawa Olhovchenkov wrote: On Wed, Feb 19, 2014 at 11:04:49PM +0200, Alexander Motin wrote: On 19.02.2014 22:04, Adrian Chadd wrote: On 19 February 2014 11:59, Alexander Motin m...@freebsd.org wrote:

[rfc] bind per-cpu timeout threads to each CPU

2014-02-18 Thread Adrian Chadd
Hi, This patch binds the per-CPU timeout swi threads to the CPU they're on. The scheduler may decide that a preempted kernel thread that's still runnable and this can happen during things like per-CPU TCP timers firing. Thanks, Index: sys/kern/kern_timeout.c