Module: xenomai-rpm
Branch: for-upstream
Commit: 3200660065146915976c193387bf0851be10d0cc

Author: Philippe Gerum <>
Date:   Sat Aug 28 17:29:42 2010 +0200

nucleus/sched: prevent remote wakeup from triggering a debug assertion

"The task that was scheduled in without XNRESCHED set locally has been
woken up by a remote CPU. The waker requeued the task and set the
resched condition for itself and in the resched proxy mask for the
remote CPU. But there is at least one place in the Xenomai code where
we drop the nklock between xnsched_set_resched and xnpod_schedule:
do_taskexit_event (I bet there are even more). Now the resched target
CPU runs into a timer handler, issues xnpod_schedule unconditionally,
and happens to find the woken-up task before it is actually informed
via an IPI."

"Yes, and whether we set the bit and call xnpod_schedule atomically
does not really matter either: the IPI takes time to propagate, and
since xnarch_send_ipi does not wait for the IPI to have been received
on the remote CPU, there is no guarantee that xnpod_schedule could not
have been called in the mean time.

More importantly, since in order to do an action on a remote
xnsched_t, we need to hold the nklock, is there any point in not
setting the XNRESCHED bit on that distant structure, at the same time
as when we set the cpu bit on the local sched structure mask and send
the IPI? This way, setting the XNRESCHED bit in the IPI handler would
no longer be necessary, and we would avoid the race."

What this patch does is having xnsched_set_resched() set the
rescheduling bit in the proxy mask from the remote CPU as well,
leaving the IPI handler set the XNRESCHED bit locally. This way:

- by setting the remote scheduling bit early on, we won't trigger the
  debug assertion under the scenario Jan discovered anymore.

- in non-debug mode, we will wait for the remote CPU to receive the
  IPI or issue a local request for rescheduling, whichever comes first
  (i.e. we won't cause xnpod_schedule() to accept useless rescheduling
  requests by raising XNRESCHED too early). The basic assumption here
  is that, since we have an IPI in flight, it does not bring much
  value to have the remote CPU reschedule for us immediately, since an
  interrupt will preempt the just resumed thread shortly.


 include/nucleus/sched.h |    3 +++
 ksrc/nucleus/pod.c      |    8 +++++++-
 2 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/include/nucleus/sched.h b/include/nucleus/sched.h
index 441a3a2..f75c6f6 100644
--- a/include/nucleus/sched.h
+++ b/include/nucleus/sched.h
@@ -184,7 +184,10 @@ static inline int xnsched_self_resched_p(struct xnsched 
 #define xnsched_set_resched(__sched__) do {                            \
   xnsched_t *current_sched = xnpod_current_sched();                    \
   xnarch_cpu_set(xnsched_cpu(__sched__), current_sched->resched);      \
+  if (unlikely(current_sched != (__sched__)))                          \
+      xnarch_cpu_set(xnsched_cpu(__sched__), (__sched__)->resched);    \
   setbits(current_sched->status, XNRESCHED);                           \
+  /* remote will set XNRESCHED locally in the IPI handler */           \
 } while (0)
 void xnsched_zombie_hooks(struct xnthread *thread);
diff --git a/ksrc/nucleus/pod.c b/ksrc/nucleus/pod.c
index 50b6d01..623bdff 100644
--- a/ksrc/nucleus/pod.c
+++ b/ksrc/nucleus/pod.c
@@ -285,7 +285,13 @@ void xnpod_schedule_handler(void) /* Called with hw 
interrupts off. */
-       xnsched_set_self_resched(sched);
+       /*
+        * xnsched_set_resched() did set the resched mask remotely. We
+        * just need to make sure that our rescheduling request won't
+        * be filtered out locally when testing for XNRESCHED
+        * presence.
+        */
+       setbits(sched->status, XNRESCHED);

Xenomai-git mailing list

Reply via email to