patch BW after patch % change
--- --
randrw1352 MB/s 2164 MB/s +60%
randwrite 1710 MB/s 2550 MB/s +49%
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/locking/rwsem-xadd.c
+ RWSEM_ACTIVE_READ_BIAS
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
arch/alpha/include/asm/rwsem.h|3 ++-
arch/ia64/include/asm/rwsem.h |2 +-
arch/s390/include/asm/rwsem.h |2 +-
arch/x86/include/asm/rwsem.h |3 ++-
include/asm-generic/rwsem.h |4 ++--
i
Move the rwsem_down_read_failed() function down to below the
optimistic spinning section before enabling optimistic spinning for
the readers. It is because the rwsem_down_read_failed() function will
call rwsem_optimistic_spin() in later patch.
There is no change in code.
Signed-off-by: Waiman
This patch modifies rwsem_spin_on_owner() to return a tri-state value
to better reflect the state of lock holder which enables us to make a
better decision of what to do next.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/locking/rwsem-xadd.c | 14 +-
1 files c
.
Both the spinning threshold and the default value for rspin_enabled
can be overridden by architecture specific rwsem.h header file.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
include/linux/rwsem.h | 19 +++-
kernel/locking/rwsem-xadd.c
purpose than
is actually needed.
Suggested-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/locking/osq_lock.c |7 ++-
1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/kernel/locking/osq_lock.c b/ke
--- --
randrw1210 MB/s 1352 MB/s +12%
randwrite 1622 MB/s 1710 MB/s +5.4%
The write-only microbench also showed improvement because some read
locking was done by the XFS code.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
ed-off-by: Waiman Long <waiman.l...@hpe.com>
---
arch/alpha/include/asm/rwsem.h|8 +---
arch/ia64/include/asm/rwsem.h |7 ++-
arch/s390/include/asm/rwsem.h |7 +--
arch/x86/include/asm/rwsem.h | 19 +--
include/asm-generic/rwsem.h
of different systems as well as for
testing purposes.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
Documentation/kernel-parameters.txt |3 +++
kernel/locking/rwsem-xadd.c | 14 +-
2 files changed, 16 insertions(+), 1 deletions(-)
diff --git a/Documen
This patch adds a new document file on how to use the TP futexes.
Signed-off-by: Waiman Long <long...@redhat.com>
---
Documentation/00-INDEX | 2 +
Documentation/tp-futex.txt | 161 +
2 files changed, 163 insertions(+)
create mode
the lock is acquired.
2) Bits 08-15: reserved
3) Bits 16-30: how many time the task sleeps in the optimistic
spinning loop.
By returning the TP status code, an external monitoring or tracking
program can have a macro view of how the TP futexes are performing.
Signed-off-by: Waiman Long <l
o link the futex state objects as well
as a new spinlock to manage them are added to the hash bucket.
To limit size increase for UP systems, these new fields are only for
SMP machines where the cacheline alignment of the hash bucket leaves
it with enough empty space for the new fields.
Signed-off-by: Waima
As the futex_state structure will be overloaded in later patches
to be used by non-PI futexes, it is necessary to add a type field to
distinguish among different types of futexes.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 15 +++
1 file chang
in the serialization mutex.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 50 ++
1 file changed, 42 insertions(+), 8 deletions(-)
diff --git a/kernel/futex.c b/kernel/futex.c
index 22f7906..91b2e02 100644
--- a/kernel/f
states are also renamed.
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/linux/sched.h | 4 +-
kernel/futex.c| 107 +-
2 files changed, 56 insertions(+), 55 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/s
erformance characteristics of the TP futexes when implementing
userspace mutex and rwlock respectively when compared with other
possible way of doing so via the wait-wake futexes.
Waiman Long (21):
perf bench: New microbenchmark for userspace mutex performance
perf bench: New microbenchmark for
%
40 15,383,536 27,687,160 +80%
50 13,290,368 23,096,937 +74%
1008,577,763 14,410,909 +68%
1us sleep 176,450 179,660 +2%
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/uapi/linux/f
-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 85 +-
1 file changed, 79 insertions(+), 6 deletions(-)
diff --git a/kernel/futex.c b/kernel/futex.c
index 6a59e6d..46a1a4b 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -
This patch extends the futex-mutex and futex-rwlock microbenchmarks to
test userspace mutexes and rwlocks built on top of the TP futexes. We
can then compare the relative performance of those userspace locks
based on different type of futexes.
Signed-off-by: Waiman Long <long...@redhat.
Unlike wake_up_process(), wake_up_q() doesn't tell us how many
tasks have been woken up. This information can sometimes be useful
for tracking purpose. So wake_up_q() is now modified to return that
information.
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/linux/sched/wake_q
On 02/03/2017 01:23 PM, valdis.kletni...@vt.edu wrote:
> On Fri, 03 Feb 2017 13:03:50 -0500, Waiman Long said:
>
>> On a 2-socket 36-core E5-2699 v3 system (HT off) running on a 4.10
>> WW futex TP f
. Lock starvation should not happen on the TP
futexes as long as the underlying kernel mutex is lock starvation
free which is the case for 4.10 and later kernel.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 136 ++
For debugging purpose, it is sometimes useful to dump the internal
states in the futex hash bucket table. This patch adds a file
"futex_hash_table" in debugfs root filesystem to dump the internal
futex states.
Signed-off-by: Waiman Long <long...@redhat.com>
---
ker
The tp-futex.txt was updated to add description about shared locking
support.
Signed-off-by: Waiman Long <long...@redhat.com>
---
Documentation/tp-futex.txt | 163 +++--
1 file changed, 143 insertions(+), 20 deletions(-)
diff --git a/Documentat
write lock/unlock code. This is by design to minimize
any additional overhead for mutex lock and unlock. As a result,
the TP futex rwlock prefers writers a bit more than readers.
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/uapi/linux/futex.h | 28 +-
ker
mechanism hasn't been
enabled yet. A new field locksteal_disabled is added to the futex
state object for controlling reader lock stealing. So the waiting
reader must retrieve the futex state object first before doing it.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.
%
Exclusive unlock futex calls = 8.9%
EAGAIN lock errors = 99.4%
Process wakeups = 0.8%
Per-thread Locking Rates:
Avg = 126,202 ops/sec (+- 0.21%)
Min = 120,114 ops/sec
Max = 131,375 ops/sec
Signed-off-by: Waiman Long <long...@redhat.com>
---
tools/perf/Documen
Two new helper functions cmpxchg_futex_value() and get_futex_value()
are added to access and change the futex value without the hash
bucket lock. As a result, page fault is enabled and the page will
be faulted in if not present yet.
Signed-off-by: Waiman Long <long...@redhat.com>
---
= 0.2%
Per-thread Locking Rates:
Avg = 75,392 ops/sec (+- 0.30%)
Min = 71,259 ops/sec
Max = 78,211 ops/sec
Signed-off-by: Waiman Long <long...@redhat.com>
---
tools/perf/Documentation/perf-bench.txt | 3 +
tools/perf/bench/bench.h| 1 +
tools/perf
Two new helper functions (task_pi_list_add & task_pi_list_del)
are created to consolidate all the pure pi_state_list addition and
insertion codes. The set_owner argument in task_pi_list_add() will
be needed in a later patch.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/fu
-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 59 --
1 file changed, 53 insertions(+), 6 deletions(-)
diff --git a/kernel/futex.c b/kernel/futex.c
index 46a1a4b..348b44c 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -63,6
.
Both the spinning threshold and the default value for rspin_enabled
can be overridden by architecture specific rwsem.h header file.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
include/linux/rwsem.h | 19 +++-
kernel/locking/rwsem-xadd.c
This patch modifies rwsem_spin_on_owner() to return a tri-state value
to better reflect the state of lock holder which enables us to make a
better decision of what to do next.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/locking/rwsem-xadd.c | 14 +-
1 files c
Move the rwsem_down_read_failed() function down to below the
optimistic spinning section before enabling optimistic spinning for
the readers. It is because the rwsem_down_read_failed() function will
call rwsem_optimistic_spin() in later patch.
There is no change in code.
Signed-off-by: Waiman
patch BW after patch % change
--- --
randrw1352 MB/s 2164 MB/s +60%
randwrite 1710 MB/s 2550 MB/s +49%
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/locking/rwsem-xadd.c
of different systems as well as for
testing purposes.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
Documentation/kernel-parameters.txt |3 +++
kernel/locking/rwsem-xadd.c | 14 +-
2 files changed, 16 insertions(+), 1 deletions(-)
diff --git a/Documen
new boot parameter to change the reader spinning
threshold which can be system specific.
Waiman Long (10):
locking/osq: Make lock/unlock proper acquire/release barrier
locking/rwsem: Stop active read lock ASAP
locking/rwsem: Make rwsem_spin_on_owner() return a tri-state value
locking/rwse
ed-off-by: Waiman Long <waiman.l...@hpe.com>
---
arch/alpha/include/asm/rwsem.h|8 +---
arch/ia64/include/asm/rwsem.h |7 ++-
arch/s390/include/asm/rwsem.h |7 +--
arch/x86/include/asm/rwsem.h | 19 +--
include/asm-generic/rwsem.h
--- --
randrw1210 MB/s 1352 MB/s +12%
randwrite 1622 MB/s 1710 MB/s +5.4%
The write-only microbench also showed improvement because some read
locking was done by the XFS code.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
+ RWSEM_ACTIVE_READ_BIAS
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
arch/alpha/include/asm/rwsem.h|3 ++-
arch/ia64/include/asm/rwsem.h |2 +-
arch/s390/include/asm/rwsem.h |2 +-
arch/x86/include/asm/rwsem.h |3 ++-
include/asm-generic/rwsem.h |4 ++--
i
. If
there are sufficient more successful spin attempts than failed ones,
it will try to reactivate reader spinning.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
include/linux/rwsem.h | 12
kernel/locking/rwsem-xadd.c | 27 +--
2 files chang
org>
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/locking/osq_lock.c | 24 ++--
1 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c
index 05a3785..3da0b97 100644
--- a/kernel/locking/osq_
, isn't quite working yet. More
investigation will have to be done to see if this can be fixed or
the timeout parameter has to be scrapped entirely.
Waiman Long (4):
futex: Add futex_set_timer() helper function
futex: Rename futex_pi_state to futex_state
futex: Throughput-optimized (TO) futexes
57.76s 1.92s
lock count 2,827,202 9,999,640 555,025
unlock count 3,027,002 9,999,64116,217
The lock and unlock counts above show the actual numbers of futex(2)
lock and unlock syscalls that were being issued.
Signed-off-by: Waiman Long <waima
On 09/23/2016 09:28 PM, Davidlohr Bueso wrote:
+#ifdef CONFIG_RT_MUTEX_SPIN_ON_OWNER
+static bool rt_mutex_spin_on_owner(struct rt_mutex *lock,
+ struct task_struct *owner)
+{
+bool ret = true;
+
+/*
+ * The last owner could have just released the lock,
+ *
On 10/05/2016 08:19 AM, Waiman Long wrote:
On 10/04/2016 03:06 PM, Davidlohr Bueso wrote:
On Thu, 18 Aug 2016, Waiman Long wrote:
The osq_lock() and osq_unlock() function may not provide the necessary
acquire and release barrier in some cases. This patch makes sure
that the proper barriers
On 10/06/2016 05:47 PM, Dave Chinner wrote:
On Thu, Oct 06, 2016 at 11:17:18AM -0700, Davidlohr Bueso wrote:
On Thu, 18 Aug 2016, Waiman Long wrote:
Currently, when down_read() fails, the active read locking isn't undone
until the rwsem_down_read_failed() function grabs the wait_lock
On 10/06/2016 01:47 AM, Davidlohr Bueso wrote:
On Wed, 05 Oct 2016, Waiman Long wrote:
diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c
index 05a3785..1e6823a 100644
--- a/kernel/locking/osq_lock.c
+++ b/kernel/locking/osq_lock.c
@@ -12,6 +12,23 @@
*/
static
On 09/22/2016 09:32 AM, Thomas Gleixner wrote:
On Tue, 6 Sep 2016, Waiman Long wrote:
+enum futex_type {
+ TYPE_PI = 0,
+ TYPE_TO,
+};
Please introduce the futex_type magic and the related changes to the pi
code in a seperate patch so it can be verified independently.
It's sad
On 09/22/2016 05:41 PM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Davidlohr Bueso wrote:
On Thu, 22 Sep 2016, Waiman Long wrote:
BTW, my initial attempt for the new futex was to use the same workflow as
the PI futexes, but use mutex which has optimistic spinning instead of
rt_mutex.
Btw
On 09/22/2016 05:31 PM, Thomas Gleixner wrote:
On Tue, 20 Sep 2016, Waiman Long wrote:
Please be more careful of your subject lines. First thing I thought was
that you add a helper which is used in later patches to find out that you
actualy consolidate duplicated code. Something like:
futex
On 09/22/2016 04:38 PM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Waiman Long wrote:
BTW, my initial attempt for the new futex was to use the same workflow as the
PI futexes, but use mutex which has optimistic spinning instead of rt_mutex.
That version can double the throughput compared
On 09/22/2016 04:26 PM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Waiman Long wrote:
On 09/22/2016 09:34 AM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Peter Zijlstra wrote:
I'd leave out the TO part entirely (or only mention it in changelogs).
That is, I'd call the futex ops: FUTEX_LOCK
On 09/21/2016 02:59 AM, Mike Galbraith wrote:
On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote:
This patch introduces a new futex implementation called
throughput-optimized (TO) futexes.
nit: 'TO' sounds way too much like timeout... TP? You even use 'to' as
shorthand for timeout
On 09/22/2016 09:23 AM, Peter Zijlstra wrote:
On Tue, Sep 20, 2016 at 09:42:41AM -0400, Waiman Long wrote:
+/*
+ * Spinning threshold before enabling lock handoff.
+ * Each sleep will decrement the threshold by 1/32 of the start value.
+ */
+#define TO_SPIN_THRESHOLD (1<< 13)
+#
On 09/22/2016 09:51 AM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Waiman Long wrote:
Yes, I will certainly send patch to update the manpages after the new futex
gets merged into the upstream kernel. Or should I do that in parallel?
It won't get merged w/o a patch to the manpage which
On 09/22/2016 09:34 AM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Peter Zijlstra wrote:
I'd leave out the TO part entirely (or only mention it in changelogs).
That is, I'd call the futex ops: FUTEX_LOCK and FUTEX_UNLOCK.
That brings me to a different question:
How is user space going to
On 09/22/2016 04:08 PM, Waiman Long wrote:
On 09/22/2016 11:11 AM, Davidlohr Bueso wrote:
On Thu, 22 Sep 2016, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Davidlohr Bueso wrote:
On Thu, 22 Sep 2016, Thomas Gleixner wrote:
> Also what's the reason that we can't do probabilistic spinn
On 09/22/2016 11:11 AM, Davidlohr Bueso wrote:
On Thu, 22 Sep 2016, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Davidlohr Bueso wrote:
On Thu, 22 Sep 2016, Thomas Gleixner wrote:
> Also what's the reason that we can't do probabilistic spinning for
> FUTEX_WAIT and have to add yet another
This patch adds a new document file on how to use the TO futexes.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
Documentation/00-INDEX |2 +
Documentation/to-futex.txt | 140
2 files changed, 142 insertions(+), 0 deletions(-)
698,318
unlock count 3,268,896 9,999,814 134
The lock and unlock counts above show the actual numbers of futex(2)
lock and unlock syscalls that were being issued.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
include/uapi/linux/futex.h |4 +
kernel/f
This patch adds the timeout support in other futex types to TO
futexes. However, the timeout isn't as precise as the other futex
types due to the fact timer expiration can't currently be detected
when the thread is waiting in the serialization mutex.
Signed-off-by: Waiman Long <waima
xes in its
calling convention, but provides better throughput than the wait-wake
futexes by encouraging lock stealing and optimistic spinning.
Waiman Long (5):
futex: Add futex_set_timer() helper function
futex: Rename futex_pi_state to futex_state
futex: Throughput-optimized (TO) futexes
fu
states are also renamed.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
include/linux/sched.h |4 +-
kernel/futex.c| 107 +
2 files changed, 56 insertions(+), 55 deletions(-)
diff --git a/include/linux/sched.h b/include
On 09/22/2016 03:49 AM, Peter Zijlstra wrote:
On Wed, Sep 21, 2016 at 07:37:34PM -0400, Waiman Long wrote:
On 09/21/2016 02:59 AM, Mike Galbraith wrote:
On Tue, 2016-09-20 at 09:42 -0400, Waiman Long wrote:
This patch introduces a new futex implementation called
throughput-optimized
On 09/22/2016 06:40 AM, Thomas Gleixner wrote:
On Tue, 6 Sep 2016, Waiman Long wrote:
This patch adds a new document file on how to use the TO futexes.
Documentation is nice, but the proper place for documenting this is the
futex(2) man page.
Thanks,
tglx
Yes, I will certainly send
On 08/24/2016 12:00 AM, Davidlohr Bueso wrote:
On Thu, 18 Aug 2016, Waiman Long wrote:
The default reader spining threshold is current set to 4096. However,
the right reader spinning threshold may vary from one system to
another and among the different architectures. This patch adds a new
A new futex_setup_timer() helper function is added to consolidate all
the hrtimer_sleeper setup code.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/futex.c | 67 +++-
1 files changed, 37 insertions(+), 30 deletions(-)
Per-thread min/sec2 106
Per-thread max/sec 498 199
% Stddev 6.26% 0.97%
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/futex.c | 55 ---
1 files chang
As the futex_state structure will be overloaded in later patches
to be used by non-PI futexes, it is necessary to add a type field to
distinguish among different types of futexes.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/futex.c | 14 ++
1 files chang
into the pi_state_list of the futex
owner before the top waiter sleeps so that if exit_pi_state_list()
is called, the wakeup will happen. The top waiter needs to remove
its futex state object from the pi_state_list of the old owner if
the ownership changes hand.
Signed-off-by: Waiman Long <waiman.l...@hpe.
. This
microbenchmark then runs the locking rate measurement tests using
either one of those mutexes or all of them consecutively.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
tools/perf/bench/Build |1 +
tools/perf/bench/bench.h |1 +
tools/perf/bench/futex-m
o link the futex state objects as well
as a new spinlock to manage them are added to the hash bucket.
To limit size increase for UP systems, these new fields are only for
SMP machines where the cacheline alignment of the hash bucket leaves
it with enough empty space for the new fields.
Signed-off-by: Waima
used to produce the
performance data to the "perf bench futex" tool. Others can then use
it to do their own evaluation.
Waiman Long (13):
futex: Consolidate duplicated timer setup code
futex: Rename futex_pi_state to futex_state
futex: Add helpers to get & cmpxchg futex value wit
Two new helper functions (task_pi_list_add & task_pi_list_del)
are created to consolidate all the pure pi_state_list addition and
insertion codes. The set_owner argument in task_pi_list_add() will
be needed in a later patch.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kern
Two new helper functions cmpxchg_futex_value() and get_futex_value()
are added to access and change the futex value without the hash
bucket lock. As a result, page fault is enabled and the page will
be faulted in if not present yet.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
the lock is acquired in the kernel.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
kernel/futex.c | 30 ++
1 files changed, 22 insertions(+), 8 deletions(-)
diff --git a/kernel/futex.c b/kernel/futex.c
index d260410..1219f32 100644
--- a/kernel/futex.c
states are also renamed.
Signed-off-by: Waiman Long <waiman.l...@hpe.com>
---
include/linux/sched.h |4 +-
kernel/futex.c| 107 +
2 files changed, 56 insertions(+), 55 deletions(-)
diff --git a/include/linux/sched.h b/include
On 09/23/2016 09:02 AM, Thomas Gleixner wrote:
On Thu, 22 Sep 2016, Waiman Long wrote:
Locking was done mostly by lock stealing. This is where most of the
performance benefit comes from, not optimistic spinning.
How does the lock latency distribution of all this look like and how fair
On 10/01/2016 02:47 AM, Thomas Gleixner wrote:
On Fri, 30 Sep 2016, Waiman Long wrote:
+ WRITE_ONCE(state->owner, current);
+ preempt_disable();
+ for (;;) {
+ ret = futex_trylock(uaddr, vpid,, true);
Did you actually read what I said? You CANNOT acc
%
40 15,383,536 27,687,160 +80%
50 13,290,368 23,096,937 +74%
1008,577,763 14,410,909 +68%
1us sleep 176,450 179,660 +2%
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/uapi/linux/f
states are also renamed.
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/linux/sched.h | 4 +-
kernel/futex.c| 107 +-
2 files changed, 56 insertions(+), 55 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/s
As the futex_state structure will be overloaded in later patches
to be used by non-PI futexes, it is necessary to add a type field to
distinguish among different types of futexes.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 15 +++
1 file chang
o link the futex state objects as well
as a new spinlock to manage them are added to the hash bucket.
To limit size increase for UP systems, these new fields are only for
SMP machines where the cacheline alignment of the hash bucket leaves
it with enough empty space for the new fields.
Signed-off-by: Waima
Two new helper functions (task_pi_list_add & task_pi_list_del)
are created to consolidate all the pure pi_state_list addition and
insertion codes. The set_owner argument in task_pi_list_add() will
be needed in a later patch.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/fu
Two new helper functions cmpxchg_futex_value() and get_futex_value()
are added to access and change the futex value without the hash
bucket lock. As a result, page fault is enabled and the page will
be faulted in if not present yet.
Signed-off-by: Waiman Long <long...@redhat.com>
---
A new futex_setup_timer() helper function is added to consolidate all
the hrtimer_sleeper setup code.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 67 --
1 file changed, 37 insertions(+), 30 deletions(-)
diff
121,129152,242 -
rwlock 99,149162,887 30,362
Patches 7 and 16 contain more detailed information about the
performance characteristics of the TP futexes when implementing
userspace mutex and rwlock respectively when compared with other
possible way of doi
-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 85 +-
1 file changed, 79 insertions(+), 6 deletions(-)
diff --git a/kernel/futex.c b/kernel/futex.c
index 2374cce..2d3ec8d 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -
the lock is acquired.
2) Bits 08-15: reserved
3) Bits 16-30: how many time the task sleeps in the optimistic
spinning loop.
By returning the TP status code, an external monitoring or tracking
program can have a macro view of how the TP futexes are performing.
Signed-off-by: Waiman Long <l
-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 59 --
1 file changed, 53 insertions(+), 6 deletions(-)
diff --git a/kernel/futex.c b/kernel/futex.c
index 2d3ec8d..711a2b4 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -63,6
. This
microbenchmark then runs the locking rate measurement tests using
either one of those mutexes or all of them consecutively.
Signed-off-by: Waiman Long <long...@redhat.com>
---
tools/perf/bench/Build | 1 +
tools/perf/bench/bench.h | 1 +
tools/perf/bench/futex-locks.c
in the serialization mutex.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 50 ++
1 file changed, 42 insertions(+), 8 deletions(-)
diff --git a/kernel/futex.c b/kernel/futex.c
index 3308cc3..429d85f 100644
--- a/kernel/f
. Lock starvation should not happen on the TP
futexes as long as the underlying kernel mutex is lock starvation
free which is the case for 4.10 and later kernel.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.c | 135 ++
The tp-futex.txt was updated to add description about shared locking
support.
Signed-off-by: Waiman Long <long...@redhat.com>
---
Documentation/tp-futex.txt | 163 +++--
1 file changed, 143 insertions(+), 20 deletions(-)
diff --git a/Documentat
Unlike wake_up_process(), wake_up_q() doesn't tell us how many
tasks have been woken up. This information can sometimes be useful
for tracking purpose. So wake_up_q() is now modified to return that
information.
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/linux/sched
mechanism hasn't been
enabled yet. A new field locksteal_disabled is added to the futex
state object for controlling reader lock stealing. So the waiting
reader must retrieve the futex state object first before doing it.
Signed-off-by: Waiman Long <long...@redhat.com>
---
kernel/futex.
respectively. This
microbenchmark then runs the locking rate measurement tests using
either one of those rwlocks or all of them consecutively.
Signed-off-by: Waiman Long <long...@redhat.com>
---
tools/perf/bench/bench.h | 1 +
tools/perf/bench/futex-locks.c
write lock/unlock code. This is by design to minimize
any additional overhead for mutex lock and unlock. As a result,
the TP futex rwlock prefers writers a bit more than readers.
Signed-off-by: Waiman Long <long...@redhat.com>
---
include/uapi/linux/futex.h | 28 +-
ker
For debugging purpose, it is sometimes useful to dump the internal
states in the futex hash bucket table. This patch adds a file
"futex_hash_table" in debugfs root filesystem to dump the internal
futex states.
Signed-off-by: Waiman Long <long...@redhat.com>
---
ker
1 - 100 of 301 matches
Mail list logo
