Author: mjg
Date: Wed Mar 1 05:06:21 2017
New Revision: 314474
URL: https://svnweb.freebsd.org/changeset/base/314474
Log:
locks: ensure proper barriers are used with atomic ops when necessary
Unclear how, but the locking routine for mutexes was using the *release*
barrier instead of acquire. This must have been either a copy-pasto or bad
completion.
Going through other uses of atomics shows no barriers in:
- upgrade routines (addressed in this patch)
- sections protected with turnstile locks - this should be fine as necessary
barriers are in the worst case provided by turnstile unlock
I would like to thank Mark Millard and andreast@ for reporting the problem and
testing previous patches before the issue got identified.
ps.
.-'---`-.
,' `.
| \
| \
\ _ \
,\ _ ,'-,/-)\
( * \ \,' ,' ,'-)
`._,) -',-')
\/ ''/
) / /
/ ,'-'
Hardware provided by: IBM LTC
Modified:
head/sys/kern/kern_rwlock.c
head/sys/kern/kern_sx.c
head/sys/sys/mutex.h
Modified: head/sys/kern/kern_rwlock.c
==============================================================================
--- head/sys/kern/kern_rwlock.c Wed Mar 1 05:05:05 2017 (r314473)
+++ head/sys/kern/kern_rwlock.c Wed Mar 1 05:06:21 2017 (r314474)
@@ -1151,7 +1151,7 @@ __rw_try_upgrade(volatile uintptr_t *c,
if (RW_READERS(v) > 1)
break;
if (!(v & RW_LOCK_WAITERS)) {
- success = atomic_cmpset_ptr(&rw->rw_lock, v, tid);
+ success = atomic_cmpset_acq_ptr(&rw->rw_lock, v, tid);
if (!success)
continue;
break;
Modified: head/sys/kern/kern_sx.c
==============================================================================
--- head/sys/kern/kern_sx.c Wed Mar 1 05:05:05 2017 (r314473)
+++ head/sys/kern/kern_sx.c Wed Mar 1 05:06:21 2017 (r314474)
@@ -410,7 +410,7 @@ sx_try_upgrade_(struct sx *sx, const cha
* we will wake up the exclusive waiters when we drop the lock.
*/
x = sx->sx_lock & SX_LOCK_EXCLUSIVE_WAITERS;
- success = atomic_cmpset_ptr(&sx->sx_lock, SX_SHARERS_LOCK(1) | x,
+ success = atomic_cmpset_acq_ptr(&sx->sx_lock, SX_SHARERS_LOCK(1) | x,
(uintptr_t)curthread | x);
LOCK_LOG_TRY("XUPGRADE", &sx->lock_object, 0, success, file, line);
if (success) {
Modified: head/sys/sys/mutex.h
==============================================================================
--- head/sys/sys/mutex.h Wed Mar 1 05:05:05 2017 (r314473)
+++ head/sys/sys/mutex.h Wed Mar 1 05:06:21 2017 (r314474)
@@ -185,7 +185,7 @@ void thread_lock_flags_(struct thread *,
atomic_cmpset_acq_ptr(&(mp)->mtx_lock, MTX_UNOWNED, (tid))
#define _mtx_obtain_lock_fetch(mp, vp, tid) \
- atomic_fcmpset_rel_ptr(&(mp)->mtx_lock, vp, (tid))
+ atomic_fcmpset_acq_ptr(&(mp)->mtx_lock, vp, (tid))
/* Try to release mtx_lock if it is unrecursed and uncontested. */
#define _mtx_release_lock(mp, tid) \
_______________________________________________
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
