Module: xenomai-head
Branch: master
Commit: d0a2e0f45a46e25adac820a3d672b23241ed2ba5

Author: Philippe Gerum <>
Date:   Sat Aug 28 16:36:23 2010 +0200

nucleus/shadow: shorten the uninterruptible path to secondary mode

Switching a thread from primary to secondary mode entails running a
significantly long code path with interrupts off, to hand over the
relaxing thread to the Linux scheduler.

Investigation on different architectures showed that such code path
was involved most of the time in latency peaks, typically when an
interrupt arrives at the very beginning of the migration sequence, and
remains blocked until the thread is fully switched out. Having RPI
enabled may increase the penalty, since pushing the relaxing thread to
the local RPI queue is part of this sequence (rpi_push).

Tracing reveals that a significant portion of the uninterruptible
sequence is actually spent running the rescheduling procedure
(xnpod_schedule). However, nothing requires us to suspend /and/ switch
out a relaxing thread atomically; actually, this is even inefficient,
since this tends to give a high priority to a thread going for less
real-time guarantees, over a real-time activity which could be started
by a pending interrupt.

This patch introduces a special handling of the XNRELAX bit condition
in xnpod_suspend_thread(), so that all locks (smp and local
interrupts) are dropped right before switching out the current thread,
to open a window for interrupt preemption. Additionally, interrupt
management is now shared between xnshadow_relax() and
xnpod_suspend_thread(), so that basic assumptions can be made on the
current interrupt state, to further reduce interrupt masking.

Best cases:

- no interrupt will be pending, so the relaxed thread will be switched
  out immediately.

- an interrupt will be pending for the Xenomai domain, performing
  time-critical duties such as waking up a real-time thread, in which
  case the latency to handle a real-time event will have been lower.

Worst case:

- an interrupt will be pending for Linux, in which case the
  rescheduling will be postponed until the interrupt pipeline has
  logged it (but not dispatched, since we will be running over the
  high priority Xenomai domain).


 include/asm-generic/system.h |    1 +
 ksrc/nucleus/pod.c           |   19 +++++++++++++++++--
 ksrc/nucleus/shadow.c        |   24 ++++++++++++++++++++----
 3 files changed, 38 insertions(+), 6 deletions(-)

diff --git a/include/asm-generic/system.h b/include/asm-generic/system.h
index a2c8fb9..4b5ce95 100644
--- a/include/asm-generic/system.h
+++ b/include/asm-generic/system.h
@@ -83,6 +83,7 @@ typedef unsigned long spl_t;
 #else /* !CONFIG_SMP */
 #define splexit(x)  rthal_local_irq_restore(x)
 #endif /* !CONFIG_SMP */
+#define splmax()    rthal_local_irq_disable()
 #define splnone()   rthal_local_irq_enable()
 #define spltest()   rthal_local_irq_test()
 #define splget(x)   rthal_local_irq_flags(x)
diff --git a/ksrc/nucleus/pod.c b/ksrc/nucleus/pod.c
index 27d0da8..df6009f 100644
--- a/ksrc/nucleus/pod.c
+++ b/ksrc/nucleus/pod.c
@@ -1456,12 +1456,27 @@ void xnpod_suspend_thread(xnthread_t *thread, xnflags_t 
                nkpod->schedhook(thread, mask);
 #endif /* __XENO_SIM__ */
-       if (thread == sched->curr)
+       if (thread == sched->curr) {
+               /*
+                * If the current thread is being relaxed, we must
+                * have been called from xnshadow_relax(), in which
+                * case we introduce an opportunity for interrupt
+                * delivery right before switching context, which
+                * shortens the uninterruptible code path.  This
+                * particular caller expects us to always return with
+                * interrupts enabled.
+                */
+               if (mask & XNRELAX) {
+                       xnlock_clear_irqon(&nklock);
+                       __xnpod_schedule(sched);
+                       return;
+               }
                 * If the thread is runnning on another CPU,
-                * xnpod_schedule will just trigger the IPI.
+                * xnpod_schedule will trigger the IPI as needed.
+       }
         * Ok, this one is an interesting corner case, which requires
diff --git a/ksrc/nucleus/shadow.c b/ksrc/nucleus/shadow.c
index 5f2a6be..76ea3ae 100644
--- a/ksrc/nucleus/shadow.c
+++ b/ksrc/nucleus/shadow.c
@@ -1146,7 +1146,6 @@ void xnshadow_relax(int notify, int reason)
        xnthread_t *thread = xnpod_current_thread();
        siginfo_t si;
        int prio;
-       spl_t s;
        XENO_BUGON(NUCLEUS, xnthread_test_state(thread, XNROOT));
@@ -1158,13 +1157,30 @@ void xnshadow_relax(int notify, int reason)
        trace_mark(xn_nucleus, shadow_gorelax, "thread %p thread_name %s",
                  thread, xnthread_name(thread));
-       splhigh(s);
+       /*
+        * If you intend to change the following interrupt-free
+        * sequence, /first/ make sure to:
+        *
+        * - read commit #d3242401b8
+        *
+        * - check the special handling of XNRELAX in
+        * xnpod_suspend_thread() when switching out the current
+        * thread, not to break basic assumptions we do there.
+        *
+        * We disable interrupts here to initiate the migration
+        * sequence, and let xnpod_suspend_thread() enable them back
+        * before returning to us.
+        */
+       splmax();
        rpi_push(thread->sched, thread);
        schedule_linux_call(LO_WAKEUP_REQ, current, 0);
        xnpod_suspend_thread(thread, XNRELAX, XN_INFINITE, XN_RELATIVE, NULL);
-       splexit(s);
+       /*
+        * As a special case when switching out a relaxed thread,
+        * interrupts have been re-enabled before returning to us. See
+        * xnpod_suspend_thread().
+        */
        if (XENO_DEBUG(NUCLEUS) && rthal_current_domain != rthal_root_domain)
                xnpod_fatal("xnshadow_relax() failed for thread %s[%d]",
                            thread->name, xnthread_user_pid(thread));

Xenomai-git mailing list

Reply via email to