Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-26 Thread Waiman Long
On 09/23/2016 09:02 AM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Waiman Long wrote: Locking was done mostly by lock stealing. This is where most of the performance benefit comes from, not optimistic spinning. How does the lock latency distribution of all this look like and how fair is the

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-26 Thread Waiman Long
On 09/23/2016 09:02 AM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Waiman Long wrote: Locking was done mostly by lock stealing. This is where most of the performance benefit comes from, not optimistic spinning. How does the lock latency distribution of all this look like and how fair is the

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-23 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Waiman Long wrote: > On 09/22/2016 04:38 PM, Thomas Gleixner wrote: > > > wait-wake futex PI futexTO futex > > > --- > > > max time3.49s50.91s 2.65s > > >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-23 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Waiman Long wrote: > On 09/22/2016 04:38 PM, Thomas Gleixner wrote: > > > wait-wake futex PI futexTO futex > > > --- > > > max time3.49s50.91s 2.65s > > >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 05:41 PM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Waiman Long wrote: BTW, my initial attempt for the new futex was to use the same workflow as the PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. Btw,

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 05:41 PM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Waiman Long wrote: BTW, my initial attempt for the new futex was to use the same workflow as the PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. Btw,

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 04:38 PM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Waiman Long wrote: BTW, my initial attempt for the new futex was to use the same workflow as the PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. That version can double the throughput compared with

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 04:38 PM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Waiman Long wrote: BTW, my initial attempt for the new futex was to use the same workflow as the PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. That version can double the throughput compared with

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Davidlohr Bueso wrote: > On Thu, 22 Sep 2016, Waiman Long wrote: > > > BTW, my initial attempt for the new futex was to use the same workflow as > > the PI futexes, but use mutex which has optimistic spinning instead of > > rt_mutex. > > Btw, Thomas, do you still have any

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Davidlohr Bueso wrote: > On Thu, 22 Sep 2016, Waiman Long wrote: > > > BTW, my initial attempt for the new futex was to use the same workflow as > > the PI futexes, but use mutex which has optimistic spinning instead of > > rt_mutex. > > Btw, Thomas, do you still have any

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Davidlohr Bueso
On Thu, 22 Sep 2016, Waiman Long wrote: BTW, my initial attempt for the new futex was to use the same workflow as the PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. Btw, Thomas, do you still have any interest pursuing this for rtmutexes from -rt into mainline?

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Davidlohr Bueso
On Thu, 22 Sep 2016, Waiman Long wrote: BTW, my initial attempt for the new futex was to use the same workflow as the PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. Btw, Thomas, do you still have any interest pursuing this for rtmutexes from -rt into mainline?

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 04:26 PM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Waiman Long wrote: On 09/22/2016 09:34 AM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Peter Zijlstra wrote: I'd leave out the TO part entirely (or only mention it in changelogs). That is, I'd call the futex ops: FUTEX_LOCK

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 04:26 PM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Waiman Long wrote: On 09/22/2016 09:34 AM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Peter Zijlstra wrote: I'd leave out the TO part entirely (or only mention it in changelogs). That is, I'd call the futex ops: FUTEX_LOCK

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Waiman Long wrote: > BTW, my initial attempt for the new futex was to use the same workflow as the > PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. > That version can double the throughput compared with PI futexes but still far > short of what can

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Waiman Long wrote: > BTW, my initial attempt for the new futex was to use the same workflow as the > PI futexes, but use mutex which has optimistic spinning instead of rt_mutex. > That version can double the throughput compared with PI futexes but still far > short of what can

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 04:08 PM, Waiman Long wrote: On 09/22/2016 11:11 AM, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: > Also what's the reason that we can't do probabilistic spinning for >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 04:08 PM, Waiman Long wrote: On 09/22/2016 11:11 AM, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: > Also what's the reason that we can't do probabilistic spinning for >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Waiman Long wrote: > On 09/22/2016 09:34 AM, Thomas Gleixner wrote: > > On Thu, 22 Sep 2016, Peter Zijlstra wrote: > > > > > > I'd leave out the TO part entirely (or only mention it in changelogs). > > > > > > That is, I'd call the futex ops: FUTEX_LOCK and FUTEX_UNLOCK. > >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Waiman Long wrote: > On 09/22/2016 09:34 AM, Thomas Gleixner wrote: > > On Thu, 22 Sep 2016, Peter Zijlstra wrote: > > > > > > I'd leave out the TO part entirely (or only mention it in changelogs). > > > > > > That is, I'd call the futex ops: FUTEX_LOCK and FUTEX_UNLOCK. > >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 11:11 AM, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: > Also what's the reason that we can't do probabilistic spinning for > FUTEX_WAIT and have to add yet another

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 11:11 AM, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: > Also what's the reason that we can't do probabilistic spinning for > FUTEX_WAIT and have to add yet another

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 09:34 AM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Peter Zijlstra wrote: I'd leave out the TO part entirely (or only mention it in changelogs). That is, I'd call the futex ops: FUTEX_LOCK and FUTEX_UNLOCK. That brings me to a different question: How is user space going to

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 09:34 AM, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Peter Zijlstra wrote: I'd leave out the TO part entirely (or only mention it in changelogs). That is, I'd call the futex ops: FUTEX_LOCK and FUTEX_UNLOCK. That brings me to a different question: How is user space going to

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 09:23 AM, Peter Zijlstra wrote: On Tue, Sep 20, 2016 at 09:42:41AM -0400, Waiman Long wrote: +/* + * Spinning threshold before enabling lock handoff. + * Each sleep will decrement the threshold by 1/32 of the start value. + */ +#define TO_SPIN_THRESHOLD (1<< 13) +#define

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 09:23 AM, Peter Zijlstra wrote: On Tue, Sep 20, 2016 at 09:42:41AM -0400, Waiman Long wrote: +/* + * Spinning threshold before enabling lock handoff. + * Each sleep will decrement the threshold by 1/32 of the start value. + */ +#define TO_SPIN_THRESHOLD (1<< 13) +#define

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Davidlohr Bueso
On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: > Also what's the reason that we can't do probabilistic spinning for > FUTEX_WAIT and have to add yet another specialized variant of futexes? Where would this

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Davidlohr Bueso
On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Davidlohr Bueso wrote: On Thu, 22 Sep 2016, Thomas Gleixner wrote: > Also what's the reason that we can't do probabilistic spinning for > FUTEX_WAIT and have to add yet another specialized variant of futexes? Where would this

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Davidlohr Bueso wrote: > On Thu, 22 Sep 2016, Thomas Gleixner wrote: > > Also what's the reason that we can't do probabilistic spinning for > > FUTEX_WAIT and have to add yet another specialized variant of futexes? > > Where would this leave the respective FUTEX_WAKE? A nop?

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Davidlohr Bueso wrote: > On Thu, 22 Sep 2016, Thomas Gleixner wrote: > > Also what's the reason that we can't do probabilistic spinning for > > FUTEX_WAIT and have to add yet another specialized variant of futexes? > > Where would this leave the respective FUTEX_WAKE? A nop?

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Davidlohr Bueso
On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Peter Zijlstra wrote: On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: > On 09/21/2016 02:59 AM, Mike Galbraith wrote: > >On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > >>This patch introduces a new futex

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Davidlohr Bueso
On Thu, 22 Sep 2016, Thomas Gleixner wrote: On Thu, 22 Sep 2016, Peter Zijlstra wrote: On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: > On 09/21/2016 02:59 AM, Mike Galbraith wrote: > >On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > >>This patch introduces a new futex

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Peter Zijlstra wrote: > On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: > > On 09/21/2016 02:59 AM, Mike Galbraith wrote: > > >On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > > >>This patch introduces a new futex implementation called > >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Thomas Gleixner
On Thu, 22 Sep 2016, Peter Zijlstra wrote: > On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: > > On 09/21/2016 02:59 AM, Mike Galbraith wrote: > > >On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > > >>This patch introduces a new futex implementation called > >

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Peter Zijlstra
On Tue, Sep 20, 2016 at 09:42:41AM -0400, Waiman Long wrote: > +/* > + * Spinning threshold before enabling lock handoff. > + * Each sleep will decrement the threshold by 1/32 of the start value. > + */ > +#define TO_SPIN_THRESHOLD(1 << 13) > +#define TO_SLEEP_DECREMENT

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Peter Zijlstra
On Tue, Sep 20, 2016 at 09:42:41AM -0400, Waiman Long wrote: > +/* > + * Spinning threshold before enabling lock handoff. > + * Each sleep will decrement the threshold by 1/32 of the start value. > + */ > +#define TO_SPIN_THRESHOLD(1 << 13) > +#define TO_SLEEP_DECREMENT

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 03:49 AM, Peter Zijlstra wrote: On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: On 09/21/2016 02:59 AM, Mike Galbraith wrote: On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: This patch introduces a new futex implementation called throughput-optimized (TO)

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Waiman Long
On 09/22/2016 03:49 AM, Peter Zijlstra wrote: On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: On 09/21/2016 02:59 AM, Mike Galbraith wrote: On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: This patch introduces a new futex implementation called throughput-optimized (TO)

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Peter Zijlstra
On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: > On 09/21/2016 02:59 AM, Mike Galbraith wrote: > >On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > >>This patch introduces a new futex implementation called > >>throughput-optimized (TO) futexes. > >nit: 'TO' sounds way too much

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-22 Thread Peter Zijlstra
On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote: > On 09/21/2016 02:59 AM, Mike Galbraith wrote: > >On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > >>This patch introduces a new futex implementation called > >>throughput-optimized (TO) futexes. > >nit: 'TO' sounds way too much

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-21 Thread Waiman Long
On 09/21/2016 02:59 AM, Mike Galbraith wrote: On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: This patch introduces a new futex implementation called throughput-optimized (TO) futexes. nit: 'TO' sounds way too much like timeout... TP? You even use 'to' as shorthand for timeout in the

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-21 Thread Waiman Long
On 09/21/2016 02:59 AM, Mike Galbraith wrote: On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: This patch introduces a new futex implementation called throughput-optimized (TO) futexes. nit: 'TO' sounds way too much like timeout... TP? You even use 'to' as shorthand for timeout in the

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-21 Thread Mike Galbraith
On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > This patch introduces a new futex implementation called > throughput-optimized (TO) futexes. nit: 'TO' sounds way too much like timeout... TP? You even use 'to' as shorthand for timeout in the next patch. > /* > ->>

Re: [RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-21 Thread Mike Galbraith
On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote: > This patch introduces a new futex implementation called > throughput-optimized (TO) futexes. nit: 'TO' sounds way too much like timeout... TP? You even use 'to' as shorthand for timeout in the next patch. > /* > ->>

[RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-20 Thread Waiman Long
This patch introduces a new futex implementation called throughput-optimized (TO) futexes. The goal of this new futex type is to maximize locking throughput at the expense of fairness and deterministic latency. Its throughput is higher than that of the wait-wake futexes especially on systems with

[RFC PATCH v2 3/5] futex: Throughput-optimized (TO) futexes

2016-09-20 Thread Waiman Long
This patch introduces a new futex implementation called throughput-optimized (TO) futexes. The goal of this new futex type is to maximize locking throughput at the expense of fairness and deterministic latency. Its throughput is higher than that of the wait-wake futexes especially on systems with