Re: 2.6.24.2-rt2

2008-02-26 Thread Jan Kiszka
Steven Rostedt wrote:
 We are pleased to announce the 2.6.24.2-rt2 tree, which can be
 downloaded from the location:
 
   http://rt.et.redhat.com/download/
 
 Information on the RT patch can be found at:
 
   http://rt.wiki.kernel.org/index.php/Main_Page
 
 Changes since 2.6.24-rt1
 
   - ported to 2.6.24.2
 
   - *** New ftrace utility ***
   The old latency_tracer has now been replaced with the cleaned up
   version that is being prepared for mainline.
 
   - compiler warning fix (Shi Weihua)

This important fix is still missing:

http://lkml.org/lkml/2008/2/5/295


At this chance: We still see the same unbalanced sched-other load on our
NUMA box as Gernot once reported [1]:

top - 11:19:20 up 4 min,  1 user,  load average: 29.52, 9.54, 3.37
Tasks: 502 total,  41 running, 461 sleeping,   0 stopped,   0 zombie
Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65513284k total,  1032032k used, 64481252k free, 6444k buffers
Swap:  3204896k total,0k used,  3204896k free,37312k cached

 PR   PID  NI  VIRT  SHR S %CPU %MEMTIME+  COMMAND
 20  5603   0  705m  464 R  100  0.3   1:18.19 load-balance-te
 20  5604   0  705m  464 R  100  0.3   1:18.16 load-balance-te
 20  5605   0  705m  464 R  100  0.3   1:18.18 load-balance-te
 20  5608   0  705m  464 R  100  0.3   1:18.18 load-balance-te
 20  5611   0  705m  464 R   25  0.3   0:19.58 load-balance-te
 20  5620   0  705m  464 R   25  0.3   0:19.54 load-balance-te
 20  5606   0  705m  464 R   25  0.3   0:19.56 load-balance-te
 20  5616   0  705m  464 R   25  0.3   0:19.54 load-balance-te
 20  5607   0  705m  464 R   20  0.3   0:15.64 load-balance-te
 20  5609   0  705m  464 R   20  0.3   0:15.66 load-balance-te
 20  5614   0  705m  464 R   20  0.3   0:15.68 load-balance-te
 20  5617   0  705m  464 R   20  0.3   0:15.64 load-balance-te
 20  5619   0  705m  464 R   20  0.3   0:15.64 load-balance-te
 20  5610   0  705m  464 R   17  0.3   0:13.10 load-balance-te
 20  5618   0  705m  464 R   17  0.3   0:13.04 load-balance-te
 20  5621   0  705m  464 R   17  0.3   0:13.02 load-balance-te
 20  5622   0  705m  464 R   17  0.3   0:13.02 load-balance-te
 20  5623   0  705m  464 R   17  0.3   0:13.06 load-balance-te
 20  5624   0  705m  464 R   17  0.3   0:13.02 load-balance-te
 20  5615   0  705m  464 R5  0.3   0:03.76 load-balance-te
 20  5633   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5634   0  705m  464 R5  0.3   0:03.82 load-balance-te
 20  5635   0  705m  464 R5  0.3   0:03.74 load-balance-te
 20  5636   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5638   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5640   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5612   0  705m  464 R5  0.3   0:03.78 load-balance-te
 20  5613   0  705m  464 R5  0.3   0:03.78 load-balance-te
 20  5625   0  705m  464 R5  0.3   0:03.74 load-balance-te
 20  5626   0  705m  464 R5  0.3   0:03.74 load-balance-te
 20  5627   0  705m  464 R5  0.3   0:03.82 load-balance-te
 20  5628   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5629   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5630   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5631   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5632   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5637   0  705m  464 R5  0.3   0:03.72 load-balance-te
 20  5639   0  705m  464 R5  0.3   0:03.74 load-balance-te
 20  5641   0  705m  464 R5  0.3   0:03.70 load-balance-te
 20  5642   0  705m  464 R5  0.3   0:03.80 load-balance-te

I just accidentally compiled the kernel with NR_CPUS=8 first, and then
the load was balanced. Weird.

Some time ago I tried to dig into this but didn't get far due to the
limited 

Re: 2.6.24.2-rt2

2008-02-26 Thread Jan Kiszka
Jan Kiszka wrote:
 At this chance: We still see the same unbalanced sched-other load on our
 NUMA box as Gernot once reported [1]:
 
 top - 11:19:20 up 4 min,  1 user,  load average: 29.52, 9.54, 3.37
 Tasks: 502 total,  41 running, 461 sleeping,   0 stopped,   0 zombie
 Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu4  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
 Mem:  65513284k total,  1032032k used, 64481252k free, 6444k buffers
 Swap:  3204896k total,0k used,  3204896k free,37312k cached
 

ETOOMANYKERNELS, this was from 2.6.23.12-rt14. 2.6.24.2-rt2 shows a
different patter under identical load:

top - 12:55:27 up 2 min,  1 user,  load average: 9.97, 2.42, 0.81
Tasks: 491 total,  42 running, 449 sleeping,   0 stopped,   0 zombie
Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  : 99.7%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  :  0.3%us,  0.3%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65512480k total,   580704k used, 64931776k free, 8964k buffers
Swap:  3204896k total,0k used,  3204896k free,   129720k cached

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
-
To unsubscribe from this list: send the line unsubscribe linux-rt-users in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: 2.6.24.2-rt2

2008-02-26 Thread Jan Kiszka
Steven Rostedt wrote:
 
 On Tue, 26 Feb 2008, Jan Kiszka wrote:
 What's the NUMA topology?
 4 nodes. I'm not sure if it is really NUMA related, but the same kernel
 runs that test as expected on a non-NUMA 2x2 box.

 What tasks are running, and at what priorities?
 40 pthreads, created with default parameters from a main thread which
 runs with default parameters as well. The threads simply run endless loops.

 Those three idle CPUS, should they have tasks running on them?
 For sure, given the overload situation of the system (40x full load vs.
 16 cores). Neither did we fiddle with any parameter of the system
 (knowingly, its a standard openSUSE 10.3 underneath) nor did we set
 thread affinities.

 
 Do you get different behaviour with 2.6.24.2?

Last time I checked mainline (I think 2.6.24), it was fine. It was
definitely fine for 2.6.23. But I'm going to revalidate this once the
machine is free again (tomorrow).

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
-
To unsubscribe from this list: send the line unsubscribe linux-rt-users in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Markers: multi-probe locking fun (was: Re: [PATCH 0/2] Markers Implementation for RCU Tracing - Ver II)

2008-02-19 Thread Jan Kiszka
Paul E. McKenney wrote:
 On Mon, Feb 18, 2008 at 01:47:31PM +0100, Jan Kiszka wrote:
 K. Prasad wrote:
 Hi Ingo,
 Please accept these patches into the rt tree which convert the
 existing RCU tracing mechanism for Preempt RCU and RCU Boost into
 markers.
  
 These patches are based upon the 2.6.24-rc5-rt1 kernel tree.
  
 Along with marker transition, the RCU Tracing infrastructure has also
 been modularised to be built as a kernel module, thereby enabling
 runtime changes to the RCU Tracing infrastructure.
  
 Patch [1/2] - Patch that converts the Preempt RCU tracing in
 rcupreempt.c into markers.
  
 Patch [1/2] - Patch that converts the Preempt RCU Boost tracing in
 rcupreempt-boost.c into markers.
  
 I have a technical problem with marker-based RCU tracing: It causes
 nasty recursions with latest multi-probe marker patches (sorry, no link
 at hand, can be found in latest LTTng, maybe also already in -mm). Those
 patches introduce a marker probe trampoline like this:

 void marker_probe_cb(const struct marker *mdata, void *call_private,
  const char *fmt, ...)
 {
  va_list args;
  char ptype;

  /*
   * rcu_read_lock does two things : disabling preemption to make sure the
   * teardown of the callbacks can be done correctly when they are in
   * modules and they insure RCU read coherency.
   */
  rcu_read_lock();
  preempt_disable();
  ...

 Can we do multi-probe with pure preempt_disable/enable protection? I
 guess it's fine with classic RCU, but what about preemptible RCU? Any
 suggestion appreciated!
 
 If you substitute synchronize_sched() for synchronize_rcu(), this should
 work fine.  Of course, this approach would cause RCU tracing to degrade
 latencies somewhat in -rt.
 
 If tracing is using call_rcu(), we will need to add a call_sched()
 or some such.

You mean something like #define call_sched call_rcu_classic?

I just learned that there is another reason for killing
rcu_read_lockfriends from the marker probes: It can deadlock on -rt
with PREEMPT_RCU_BOOST (hit probe inside rq-lock protected region =
rcu_read_unlock triggers unboost = stuck on rq_lock :( ).

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
-
To unsubscribe from this list: send the line unsubscribe linux-rt-users in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 0/2] Markers Implementation for RCU Tracing - Ver II

2008-02-18 Thread Jan Kiszka
K. Prasad wrote:
 Hi Ingo,
   Please accept these patches into the rt tree which convert the
 existing RCU tracing mechanism for Preempt RCU and RCU Boost into
 markers.
  
 These patches are based upon the 2.6.24-rc5-rt1 kernel tree.
  
 Along with marker transition, the RCU Tracing infrastructure has also
 been modularised to be built as a kernel module, thereby enabling
 runtime changes to the RCU Tracing infrastructure.
  
 Patch [1/2] - Patch that converts the Preempt RCU tracing in
 rcupreempt.c into markers.
  
 Patch [1/2] - Patch that converts the Preempt RCU Boost tracing in
 rcupreempt-boost.c into markers.
  

I have a technical problem with marker-based RCU tracing: It causes
nasty recursions with latest multi-probe marker patches (sorry, no link
at hand, can be found in latest LTTng, maybe also already in -mm). Those
patches introduce a marker probe trampoline like this:

void marker_probe_cb(const struct marker *mdata, void *call_private,
const char *fmt, ...)
{
va_list args;
char ptype;

/*
 * rcu_read_lock does two things : disabling preemption to make sure the
 * teardown of the callbacks can be done correctly when they are in
 * modules and they insure RCU read coherency.
 */
rcu_read_lock();
preempt_disable();
...

Can we do multi-probe with pure preempt_disable/enable protection? I
guess it's fine with classic RCU, but what about preemptible RCU? Any
suggestion appreciated!

Jan

PS: You will run into this issue if you try to marry latest -rt with
latest LTTng. Straightforward workaround is to comment-out any RCU
trace_mark occurrences.

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
-
To unsubscribe from this list: send the line unsubscribe linux-rt-users in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 0/2] Markers Implementation for RCU Tracing - Ver II

2008-02-18 Thread Jan Kiszka
K. Prasad wrote:
 Hi Ingo,
   Please accept these patches into the rt tree which convert the
 existing RCU tracing mechanism for Preempt RCU and RCU Boost into
 markers.
  
 These patches are based upon the 2.6.24-rc5-rt1 kernel tree.
  
 Along with marker transition, the RCU Tracing infrastructure has also
 been modularised to be built as a kernel module, thereby enabling
 runtime changes to the RCU Tracing infrastructure.
  
 Patch [1/2] - Patch that converts the Preempt RCU tracing in
 rcupreempt.c into markers.
  
 Patch [1/2] - Patch that converts the Preempt RCU Boost tracing in
 rcupreempt-boost.c into markers.
  
 Thanks,
 K.Prasad
 ([EMAIL PROTECTED])

The correct marker annotation for no arguments is MARK_NOARGS.

Signed-off-by: Jan Kiszka [EMAIL PROTECTED]

---
 kernel/rcupreempt-boost.c |   24 
 kernel/rcupreempt.c   |   42 +-
 2 files changed, 33 insertions(+), 33 deletions(-)

Index: b/kernel/rcupreempt-boost.c
===
--- a/kernel/rcupreempt-boost.c
+++ b/kernel/rcupreempt-boost.c
@@ -57,10 +57,10 @@ static void rcu_boost_task(struct task_s
WARN_ON(!irqs_disabled());
WARN_ON_SMP(!spin_is_locked(task-pi_lock));
 
-   trace_mark(task_boost_called, NULL);
+   trace_mark(task_boost_called, MARK_NOARGS);
 
if (task-rcu_prio  task-prio) {
-   trace_mark(task_boosted, NULL);
+   trace_mark(task_boosted, MARK_NOARGS);
task_setprio(task, task-rcu_prio);
}
 }
@@ -84,7 +84,7 @@ void __rcu_preempt_boost(void)
 
WARN_ON(!current-rcu_read_lock_nesting);
 
-   trace_mark(boost_called, NULL);
+   trace_mark(boost_called, MARK_NOARGS);
 
/* check to see if we are already boosted */
if (unlikely(rcu_is_boosted(curr)))
@@ -102,7 +102,7 @@ void __rcu_preempt_boost(void)
 
curr-rcub_rbdp = rbd;
 
-   trace_mark(try_boost, NULL);
+   trace_mark(try_boost, MARK_NOARGS);
 
prio = rt_mutex_getprio(curr);
 
@@ -111,7 +111,7 @@ void __rcu_preempt_boost(void)
if (prio = rbd-rbs_prio)
goto out;
 
-   trace_mark(boosted, NULL);
+   trace_mark(boosted, MARK_NOARGS);
 
curr-rcu_prio = rbd-rbs_prio;
rcu_boost_task(curr);
@@ -136,7 +136,7 @@ void __rcu_preempt_unboost(void)
int prio;
unsigned long flags;
 
-   trace_mark(unboost_called, NULL);
+   trace_mark(unboost_called, MARK_NOARGS);
 
/* if not boosted, then ignore */
if (likely(!rcu_is_boosted(curr)))
@@ -174,7 +174,7 @@ void __rcu_preempt_unboost(void)
 
list_del_init(curr-rcub_entry);
 
-   trace_mark(unboosted, NULL);
+   trace_mark(unboosted, MARK_NOARGS);
 
curr-rcu_prio = MAX_PRIO;
 
@@ -235,7 +235,7 @@ static int __rcu_boost_readers(struct rc
 * Another task may have taken over.
 */
if (curr-rcu_preempt_counter != rcu_boost_counter) {
-   trace_mark(over_taken, NULL);
+   trace_mark(over_taken, MARK_NOARGS);
return 1;
}
 
@@ -266,7 +266,7 @@ void rcu_boost_readers(void)
 
prio = rt_mutex_getprio(curr);
 
-   trace_mark(try_boost_readers, NULL);
+   trace_mark(try_boost_readers, MARK_NOARGS);
 
if (prio = rcu_boost_prio) {
/* already boosted */
@@ -276,7 +276,7 @@ void rcu_boost_readers(void)
 
rcu_boost_prio = prio;
 
-   trace_mark(boost_readers, NULL);
+   trace_mark(boost_readers, MARK_NOARGS);
 
/* Flag that we are the one to unboost */
curr-rcu_preempt_counter = ++rcu_boost_counter;
@@ -309,12 +309,12 @@ void rcu_unboost_readers(void)
 
spin_lock_irqsave(rcu_boost_wake_lock, flags);
 
-   trace_mark(try_unboost_readers, NULL);
+   trace_mark(try_unboost_readers, MARK_NOARGS);
 
if (current-rcu_preempt_counter != rcu_boost_counter)
goto out;
 
-   trace_mark(unboost_readers, NULL);
+   trace_mark(unboost_readers, MARK_NOARGS);
 
/*
 * We could also put in something that
Index: b/kernel/rcupreempt.c
===
--- a/kernel/rcupreempt.c
+++ b/kernel/rcupreempt.c
@@ -308,7 +308,7 @@ static void __rcu_advance_callbacks(stru
if (rdp-waitlist[GP_STAGES - 1] != NULL) {
*rdp-donetail = rdp-waitlist[GP_STAGES - 1];
rdp-donetail = rdp-waittail[GP_STAGES - 1];
-   trace_mark(rcupreempt_trace_move2done, NULL);
+   trace_mark(rcupreempt_trace_move2done, MARK_NOARGS);
}
for (i = GP_STAGES - 2; i = 0; i--) {
if (rdp-waitlist[i] != NULL) {
@@ -327,7 +327,7 @@ static void __rcu_advance_callbacks(stru
wlc

Re: KVM and Prempt?

2007-10-23 Thread Jan Kiszka
Sven-Thorsten Dietrich wrote:
 On Mon, 2007-10-22 at 09:01 +0200, Back, Michael (ext) wrote:
 Hallo,
 I tried to run Windows XP with KVM on Linux 2.6.31.1 on a 
 
 You mean .21.1 ? 

Classic typo I interestingly also did several times the last week. :)

 
 AMD Opteron and on a Intel Xeon, on both it works fine!
 
 After this test I patch the kernel with the current prempt-patch and on
 both it doesn't works! 
 
 Did you try against 2.6.23-rt1.

kvm in -rt1 is not usable. It's too old, lacking PREEMPT_NOTIFIER
support, thus quickly triggering lockdep.

 
 If you must stay on .21, you might have some other issues with the AMD
 and NUMA.
 
 At the very least, you will need to apply the attached patch from git
 somehow, although this patch is against a new scheduler post 2.6.22, so
 good luck :)

--snip--

Those patches are already mainline... :-

What you rather need are latest kvm patches, or - if building the kvm
distribution out of tree - a patch to enabled CONFIG_PREEMPT_NOTIFIERS
unconditionally:

--- linux-2.6.23.1-rt/kernel/Kconfig.preempt.orig
+++ linux-2.6.23.1-rt/kernel/Kconfig.preempt
@@ -136,6 +136,7 @@

 config PREEMPT_NOTIFIERS
bool
+   default y

 config PREEMPT_BKL
bool


Still, I'm seeing oopses here (more precisely, lock validator
complaints), but I need to re-test, better using kvm from git instead of
kvm-48.

Beyond this, I'm struggling to understand 300-400 us vm-exit latencies
(over Intel VMX), which appear to be independent of the underlying
system. See kvm-devel. Such latencies would limit the RT usability of
kvm - unless you spent dedicated CPUs.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
-
To unsubscribe from this list: send the line unsubscribe linux-rt-users in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: periodic bursts

2007-10-01 Thread Jan Kiszka

Kaya, Sinan wrote:

Hello all,
My realtime system experiences periodic bursts every 4 hours. How can
i find the reason for this  ? I know the traceit tool but it dumps
so much data and it is useless under this case. I need to find what 
happened at the exact time of burst.


The trace-it tool is a demo, you need to adopt it to your scenario, 
specifically make it trigger the stop once you detected some burst. 
Means, you need to put its code into your application. Did you do this?




I use rtai_smi module to disable SMI interrupts


Then you will probably like this tool even more:
http://www.rts.uni-hannover.de/rtaddon/#smictrl


and set my network interface driver priorities to 99. So it should be
something non maskable but what ?


Out-of-the-box NIC drivers are generally not suited for more than
soft-RT. They allocate memory (skbs) from global pools which takes
varying time (or may even fail if your are short on memory), sometimes
they try to pile up packets first before they raise an IRQ, and they
often contain hardware/link state watchdogs that can inject latencies 
right at the wrong time. If you need low latency by design, more work is 
required.


In any case, understanding this particular problem comes first, and
collecting the right traces will help.

Jan

--
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
-
To unsubscribe from this list: send the line unsubscribe linux-rt-users in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html