Author: jeff
Date: Thu Jan 23 03:36:50 2020
New Revision: 357014
URL: https://svnweb.freebsd.org/changeset/base/357014
Log:
Block the thread lock in sched_throw() and use cpu_switch() to unblock
it. The introduction of lockless switch in r355784 created a race to
re-use the exiting thread that was only possible to hit on a hypervisor.
Reported/Tested by: rlibby
Discussed with: rlibby, jhb
Modified:
head/sys/kern/sched_ule.c
Modified: head/sys/kern/sched_ule.c
==============================================================================
--- head/sys/kern/sched_ule.c Thu Jan 23 01:49:22 2020 (r357013)
+++ head/sys/kern/sched_ule.c Thu Jan 23 03:36:50 2020 (r357014)
@@ -2894,7 +2894,7 @@ sched_throw(struct thread *td)
struct thread *newtd;
struct tdq *tdq;
- if (td == NULL) {
+ if (__predict_false(td == NULL)) {
#ifdef SMP
PCPU_SET(sched, DPCPU_PTR(tdq));
#endif
@@ -2912,13 +2912,18 @@ sched_throw(struct thread *td)
tdq_load_rem(tdq, td);
td->td_lastcpu = td->td_oncpu;
td->td_oncpu = NOCPU;
+ thread_lock_block(td);
}
newtd = choosethread();
spinlock_enter();
TDQ_UNLOCK(tdq);
KASSERT(curthread->td_md.md_spinlock_count == 1,
("invalid count %d", curthread->td_md.md_spinlock_count));
- cpu_throw(td, newtd); /* doesn't return */
+ /* doesn't return */
+ if (__predict_false(td == NULL))
+ cpu_throw(td, newtd); /* doesn't return */
+ else
+ cpu_switch(td, newtd, TDQ_LOCKPTR(tdq));
}
/*
_______________________________________________
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
