[Xenomai-core] kernel BUG running cyclictest on powerpc 405
I have built a 2.6.23-rc7 kernel (from Denx git) with Xenomai 2.4-rc3. Architecture is powerpc, processor is a 405GP. I had to make some additions to arch/powerpc/kernel/head_40x.S, and I can submit a patch if someone tells me where to post it. However, while the kernel boots and runs, the xenomai user apps do not work correctly. Specifically, when running cyclictest, I get the following: # ./cyclictest 0.00 0.00 0.00 1/18 668 T: 0 ( 668) P:99 I: [ cut here ] 0 Avg: 0 Max:-100 1000 C: 5kernel BUG at /n/maytag/home/sfalco/linux/embedded/ppc3/linux/linux-2.6-denx/kernel/ipipe/core.c:314! Min: 24 AcOops: Exception in kernel mode, sig: 5 [#1] Netdec Modules linked in: NIP: c0045dd0 LR: c0015750 CTR: c00121fc REGS: c0dbde00 TRAP: 0700 Not tainted (2.6.23-rc7) MSR: 00021030 ME,IR,DR CR: 22000422 XER: 2000 TASK = c0fe7420[668] 'cyclictest' THREAD: c0dbc000 GPR00: 4900 c0dbdeb0 c0fe7420 003f c00115dc GPR08: 48ff c0296090 0001 0001 1001c23c 00ff9900 0001 GPR16: 7fbabcb0 10015090 10015068 0ff81940 029a 0ff815d0 GPR24: c02a2fcc 0280 c030c200 001f 003f 0010 c028e3bc NIP [c0045dd0] __ipipe_restore_root+0x1c/0x50 LR [c0015750] uic_mask_irq+0x84/0x98 Call Trace: [c0dbdeb0] [c0054cdc] hisyscall_event+0x1fc/0x310 (unreliable) [c0dbded0] [c00439fc] __ipipe_ack_level_irq+0x50/0x78 [c0dbdee0] [c0007b2c] __ipipe_ack_irq+0x28/0x3c [c0dbdef0] [c0007e68] __ipipe_handle_irq+0x12c/0x148 [c0dbdf20] [c00080f8] __ipipe_grab_irq+0x3c/0xc0 [c0dbdf40] [c000e338] __ipipe_ret_from_except+0x0/0xc Instruction dump: 0.00 0.00 0.00 2/18 66Xenomai: POSIX: destroyed thread c0f803203d20c02c 8 004a78 3120 7d690110 0f0b 2f83 3d60c029 392b6050 t: 39 Avg: 29 Max: 39 T: 0 ( 668) P:99 I:1000 C: 8 Min: 24 Act: 24 Avg: 29 Max: 39 Trace/breakpoint trap The bug is coming from ipipe/core.c:314, which is the BUG_ON here: void __ipipe_restore_root(unsigned long x) { BUG_ON(!ipipe_root_domain_p); if (x) __ipipe_stall_root(); else __ipipe_unstall_root(); } How do I debug this? I have done a little kernel hacking, but I am a noob when it comes to Xenomai... Steve ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] kernel BUG running cyclictest on powerpc 405
Steven A. Falco wrote: I have built a 2.6.23-rc7 kernel (from Denx git) with Xenomai 2.4-rc3. Architecture is powerpc, processor is a 405GP. I had to make some additions to arch/powerpc/kernel/head_40x.S, and I can submit a patch if someone tells me where to post it. Here would be nice, or to the Adeos list, indifferently - I would pick it from both anyway. However, while the kernel boots and runs, the xenomai user apps do not work correctly. Specifically, when running cyclictest, I get the following: This patch should fix this issue. diff --git a/arch/powerpc/sysdev/uic.c b/arch/powerpc/sysdev/uic.c index eeb38e2..5a38086 100644 --- a/arch/powerpc/sysdev/uic.c +++ b/arch/powerpc/sysdev/uic.c @@ -48,7 +48,7 @@ struct uic { int index; int dcrbase; - spinlock_t lock; + ipipe_spinlock_t lock; /* The remapper for this UIC */ struct irq_host *irqhost; How do I debug this? I have done a little kernel hacking, but I am a noob when it comes to Xenomai... Well, actually, it's shame on me. I did not notice that the universal interrupt controller code was missing some bits in post-2.6.21 I-pipe patches. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] kernel BUG running cyclictest on powerpc 405
Thanks! I'll try your patch and report if I still have trouble. I've attached my patch for PPC405. Steve Philippe Gerum wrote: Steven A. Falco wrote: I have built a 2.6.23-rc7 kernel (from Denx git) with Xenomai 2.4-rc3. Architecture is powerpc, processor is a 405GP. I had to make some additions to arch/powerpc/kernel/head_40x.S, and I can submit a patch if someone tells me where to post it. Here would be nice, or to the Adeos list, indifferently - I would pick it from both anyway. However, while the kernel boots and runs, the xenomai user apps do not work correctly. Specifically, when running cyclictest, I get the following: This patch should fix this issue. diff --git a/arch/powerpc/sysdev/uic.c b/arch/powerpc/sysdev/uic.c index eeb38e2..5a38086 100644 --- a/arch/powerpc/sysdev/uic.c +++ b/arch/powerpc/sysdev/uic.c @@ -48,7 +48,7 @@ struct uic { int index; int dcrbase; - spinlock_t lock; + ipipe_spinlock_t lock; /* The remapper for this UIC */ struct irq_host *irqhost; How do I debug this? I have done a little kernel hacking, but I am a noob when it comes to Xenomai... Well, actually, it's shame on me. I did not notice that the universal interrupt controller code was missing some bits in post-2.6.21 I-pipe patches. --- linux-2.6-denx.xenomai/arch/powerpc/kernel/head_40x.S 2007-09-28 13:40:46.0 -0400 +++ linux-2.6-denx/arch/powerpc/kernel/head_40x.S 2007-10-24 14:02:23.0 -0400 @@ -227,6 +227,12 @@ EXC_XFER_TEMPLATE(hdlr, n, MSR_KERNEL, NOCOPY, transfer_to_handler_full, \ ret_from_except_full) +#ifdef CONFIG_IPIPE +#define EXC_XFER_IPIPE(n, hdlr) \ + EXC_XFER_TEMPLATE(hdlr, n+1, MSR_KERNEL, NOCOPY, transfer_to_handler, \ + __ipipe_ret_from_except) +#endif /* CONFIG_IPIPE */ + #define EXC_XFER_LITE(n, hdlr) \ EXC_XFER_TEMPLATE(hdlr, n+1, MSR_KERNEL, NOCOPY, transfer_to_handler, \ ret_from_except) @@ -395,7 +401,11 @@ EXC_XFER_EE_LITE(0x400, handle_page_fault) /* 0x0500 - External Interrupt Exception */ +#ifdef CONFIG_IPIPE + EXCEPTION(0x0500, HardwareInterrupt, __ipipe_grab_irq, EXC_XFER_IPIPE) +#else /* !CONFIG_IPIPE */ EXCEPTION(0x0500, HardwareInterrupt, do_IRQ, EXC_XFER_LITE) +#endif /* CONFIG_IPIPE */ /* 0x0600 - Alignment Exception */ START_EXCEPTION(0x0600, Alignment) @@ -433,7 +443,11 @@ lis r0,[EMAIL PROTECTED] mtspr SPRN_TSR,r0 /* Clear the PIT exception */ addi r3,r1,STACK_FRAME_OVERHEAD +#ifdef CONFIG_IPIPE + EXC_XFER_IPIPE(0x1000, __ipipe_grab_timer) +#else /* !CONFIG_IPIPE */ EXC_XFER_LITE(0x1000, timer_interrupt) +#endif /* CONFIG_IPIPE */ #if 0 /* NOTE: ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] kernel BUG running cyclictest on powerpc 405
Steven A. Falco wrote: Thanks! I'll try your patch and report if I still have trouble. I've attached my patch for PPC405. Thanks. You may also want to upgrade to this I-pipe patch; it should apply properly against -rc7: http://download.gna.org/adeos/patches/v2.6/powerpc/adeos-ipipe-2.6.23-powerpc-DENX-2.0-02.patch You would still need the quick fix for the UIC on top of that one, though. Steve Philippe Gerum wrote: Steven A. Falco wrote: I have built a 2.6.23-rc7 kernel (from Denx git) with Xenomai 2.4-rc3. Architecture is powerpc, processor is a 405GP. I had to make some additions to arch/powerpc/kernel/head_40x.S, and I can submit a patch if someone tells me where to post it. Here would be nice, or to the Adeos list, indifferently - I would pick it from both anyway. However, while the kernel boots and runs, the xenomai user apps do not work correctly. Specifically, when running cyclictest, I get the following: This patch should fix this issue. diff --git a/arch/powerpc/sysdev/uic.c b/arch/powerpc/sysdev/uic.c index eeb38e2..5a38086 100644 --- a/arch/powerpc/sysdev/uic.c +++ b/arch/powerpc/sysdev/uic.c @@ -48,7 +48,7 @@ struct uic { int index; int dcrbase; -spinlock_t lock; +ipipe_spinlock_t lock; /* The remapper for this UIC */ struct irq_host *irqhost; How do I debug this? I have done a little kernel hacking, but I am a noob when it comes to Xenomai... Well, actually, it's shame on me. I did not notice that the universal interrupt controller code was missing some bits in post-2.6.21 I-pipe patches. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] kernel BUG running cyclictest on powerpc 405
I upgraded to the new ipipe patch and your uic fix. Attached is a new patch I needed - without it, I get the error message: .../include/linux/ipipe.h:298: warning: 'struct mm_struct' declared inside parameter list .../include/linux/ipipe.h:298: warning: its scope is only this definition or declaration, which is probably not what you want Also, there is a problem with the uic fix. It now gives this compile-time error: CC arch/powerpc/sysdev/uic.o .../arch/powerpc/sysdev/uic.c: In function 'uic_init_one': .../arch/powerpc/sysdev/uic.c:234: error: incompatible types in assignment I believe this is caused by the definition in spinlock.h: # define spin_lock_init(lock)\ do { *(lock) = SPIN_LOCK_UNLOCKED; } while (0) The SPIN_LOCK_UNLOCKED should perhaps be IPIPE_SPIN_LOCK_UNLOCKED, but I don't know if this is the right way to fix it. Steve Philippe Gerum wrote: Steven A. Falco wrote: Thanks! I'll try your patch and report if I still have trouble. I've attached my patch for PPC405. Thanks. You may also want to upgrade to this I-pipe patch; it should apply properly against -rc7: http://download.gna.org/adeos/patches/v2.6/powerpc/adeos-ipipe-2.6.23-powerpc-DENX-2.0-02.patch You would still need the quick fix for the UIC on top of that one, though. Steve Philippe Gerum wrote: Steven A. Falco wrote: I have built a 2.6.23-rc7 kernel (from Denx git) with Xenomai 2.4-rc3. Architecture is powerpc, processor is a 405GP. I had to make some additions to arch/powerpc/kernel/head_40x.S, and I can submit a patch if someone tells me where to post it. Here would be nice, or to the Adeos list, indifferently - I would pick it from both anyway. However, while the kernel boots and runs, the xenomai user apps do not work correctly. Specifically, when running cyclictest, I get the following: This patch should fix this issue. diff --git a/arch/powerpc/sysdev/uic.c b/arch/powerpc/sysdev/uic.c index eeb38e2..5a38086 100644 --- a/arch/powerpc/sysdev/uic.c +++ b/arch/powerpc/sysdev/uic.c @@ -48,7 +48,7 @@ struct uic { int index; int dcrbase; - spinlock_t lock; + ipipe_spinlock_t lock; /* The remapper for this UIC */ struct irq_host *irqhost; How do I debug this? I have done a little kernel hacking, but I am a noob when it comes to Xenomai... Well, actually, it's shame on me. I did not notice that the universal interrupt controller code was missing some bits in post-2.6.21 I-pipe patches. --- /home/sfalco/ipipe.h2007-10-25 12:59:12.0 -0400 +++ include/linux/ipipe.h 2007-10-25 12:59:23.0 -0400 @@ -31,6 +31,8 @@ #include linux/ipipe_compat.h #include asm/ipipe.h +struct mm_struct; + #ifdef CONFIG_IPIPE #define IPIPE_VERSION_STRING IPIPE_ARCH_STRING ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Xenomai v2.4-rc4: freeze with RTAI skin, fine with other skins
Philippe Gerum wrote: Fillod Stephane wrote: For the legacy RTAI application to load, the attached patch was necessary. The patch against ksrc/skins/rtai/shm.c is somewhat defeating the purpose of a lower XNCORE_PAGE_SIZE, so a better fix might be expected. This one should prevent -EINVAL from being returned. Hopefully. Nope, it doesn't :-( Most probably because still (hdrsize + 2 * pagesize heapsize). [..] In the mean time, does anyone have a clue where to look particularly? Hard to say at this point. Since the nucleus watchdog does not trigger, you may want to try disabling X86_UP_IOAPIC while keeping X86_UP_APIC, and arm the kernel NMI watchdog on the LAPIC (nmi_watchdog=2). You may be lucky and have a backtrace after the freeze. PS: maybe enabling all the nucleus debug options would catch something too. Thanks for the ideas. It looks like part of the problem came from memory corruption (access beyond heap) in the application (not so stable app in the end, he!). I'm still experiencing random freeze when freeing resources when stopping application, but runtime is stable. To be honest, the freeze was not a perfect freeze, I got just a one line trace from do_page_fault(), with no backtrace or pointer. Unhelpful. I have updated http://xenomai.org/index.php/FAQs -- Stephane ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Proposed arch/powerpc/sysdev/uic.c patch
Steven A. Falco wrote: I applied the uic patch: diff --git a/arch/powerpc/sysdev/uic.c b/arch/powerpc/sysdev/uic.c index eeb38e2..5a38086 100644 --- a/arch/powerpc/sysdev/uic.c +++ b/arch/powerpc/sysdev/uic.c @@ -48,7 +48,7 @@ struct uic { int index; int dcrbase; -spinlock_t lock; +ipipe_spinlock_t lock; /* The remapper for this UIC */ struct irq_host*irqhost; However, this would not compile because of a type mismatch. I have added the attached patch, and it now compiles and runs. But I'm not sure if this is the right way to fix it. Comments? This will work for the purpose of running an I-pipe enabled kernel, but would fail with CONFIG_IPIPE disabled. Since we need to provide both, I'm going to work on the proper patch for fixing the issue both ways. Still, your patch will work as expected for running Xenomai. Steve ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Proposed arch/powerpc/sysdev/uic.c patch
Philippe Gerum wrote: Steven A. Falco wrote: I applied the uic patch: diff --git a/arch/powerpc/sysdev/uic.c b/arch/powerpc/sysdev/uic.c index eeb38e2..5a38086 100644 --- a/arch/powerpc/sysdev/uic.c +++ b/arch/powerpc/sysdev/uic.c @@ -48,7 +48,7 @@ struct uic { int index; int dcrbase; -spinlock_t lock; +ipipe_spinlock_t lock; /* The remapper for this UIC */ struct irq_host*irqhost; However, this would not compile because of a type mismatch. I have added the attached patch, and it now compiles and runs. But I'm not sure if this is the right way to fix it. Comments? This will work for the purpose of running an I-pipe enabled kernel, but would fail with CONFIG_IPIPE disabled. Since we need to provide both, I'm going to work on the proper patch for fixing the issue both ways. Still, your patch will work as expected for running Xenomai. To be consequent, we would have to wrap spin_lock_init just like the other operations. Suggestion (not really tested): Index: linux-2.6.23-ipipe/include/linux/spinlock.h === --- linux-2.6.23-ipipe.orig/include/linux/spinlock.h +++ linux-2.6.23-ipipe/include/linux/spinlock.h @@ -89,10 +89,22 @@ extern int __lockfunc generic__raw_read_ # include linux/spinlock_up.h #endif +#undef TYPE_EQUAL +#define TYPE_EQUAL(lock, type) \ + __builtin_types_compatible_p(typeof(lock), type *) + +#define spin_lock_init(lock) \ +do { \ + if (TYPE_EQUAL((lock), __ipipe_spinlock_t)) \ + *(__ipipe_spinlock_t *)(lock) = IPIPE_SPIN_LOCK_UNLOCKED; \ + else if (TYPE_EQUAL(lock, spinlock_t)) \ + _spin_lock_init((spinlock_t *)(lock)); \ +} while (0) + #ifdef CONFIG_DEBUG_SPINLOCK extern void __spin_lock_init(spinlock_t *lock, const char *name, struct lock_class_key *key); -# define spin_lock_init(lock) \ +# define _spin_lock_init(lock) \ do { \ static struct lock_class_key __key; \ \ @@ -100,7 +112,7 @@ do { \ } while (0) #else -# define spin_lock_init(lock) \ +# define _spin_lock_init(lock) \ do { *(lock) = SPIN_LOCK_UNLOCKED; } while (0) #endif @@ -172,10 +184,6 @@ do { \ #define read_trylock(lock) __cond_lock(lock, _read_trylock(lock)) #define write_trylock(lock)__cond_lock(lock, _write_trylock(lock)) -#undef TYPE_EQUAL -#define TYPE_EQUAL(lock, type) \ - __builtin_types_compatible_p(typeof(lock), type *) - #define PICK_SPINOP(op, lock) \ do { \ if (TYPE_EQUAL((lock), __ipipe_spinlock_t)) \ Jan signature.asc Description: OpenPGP digital signature ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Proposed arch/powerpc/sysdev/uic.c patch
Jan Kiszka wrote: Philippe Gerum wrote: Steven A. Falco wrote: I applied the uic patch: diff --git a/arch/powerpc/sysdev/uic.c b/arch/powerpc/sysdev/uic.c index eeb38e2..5a38086 100644 --- a/arch/powerpc/sysdev/uic.c +++ b/arch/powerpc/sysdev/uic.c @@ -48,7 +48,7 @@ struct uic { int index; int dcrbase; -spinlock_t lock; +ipipe_spinlock_t lock; /* The remapper for this UIC */ struct irq_host*irqhost; However, this would not compile because of a type mismatch. I have added the attached patch, and it now compiles and runs. But I'm not sure if this is the right way to fix it. Comments? This will work for the purpose of running an I-pipe enabled kernel, but would fail with CONFIG_IPIPE disabled. Since we need to provide both, I'm going to work on the proper patch for fixing the issue both ways. Still, your patch will work as expected for running Xenomai. To be consequent, we would have to wrap spin_lock_init just like the other operations. Suggestion (not really tested): Thanks. I have committed a tested variant. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
[Xenomai-core] Compile-time patch for mm_struct
I needed the following patch to fix a forward reference (caught by gcc 4.1.1). I posted this as part of an earlier discussion, but I then realized that it might be overlooked, so I'm reposting it as a new thread: --- /home/sfalco/ipipe.h2007-10-25 12:59:12.0 -0400 +++ include/linux/ipipe.h 2007-10-25 12:59:23.0 -0400 @@ -31,6 +31,8 @@ #include linux/ipipe_compat.h #include asm/ipipe.h +struct mm_struct; + #ifdef CONFIG_IPIPE #define IPIPE_VERSION_STRING IPIPE_ARCH_STRING ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Compile-time patch for mm_struct
Steven A. Falco wrote: I needed the following patch to fix a forward reference (caught by gcc 4.1.1). I posted this as part of an earlier discussion, but I then realized that it might be overlooked, so I'm reposting it as a new thread: Thanks. This one was caught a few days back and only recently merged. I guess it's time to roll out a new 2.6.23/powerpc I-pipe patch. --- /home/sfalco/ipipe.h 2007-10-25 12:59:12.0 -0400 +++ include/linux/ipipe.h 2007-10-25 12:59:23.0 -0400 @@ -31,6 +31,8 @@ #include linux/ipipe_compat.h #include asm/ipipe.h +struct mm_struct; + #ifdef CONFIG_IPIPE #define IPIPE_VERSION_STRING IPIPE_ARCH_STRING ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Compile-time patch for mm_struct
Philippe Gerum wrote: Steven A. Falco wrote: I needed the following patch to fix a forward reference (caught by gcc 4.1.1). I posted this as part of an earlier discussion, but I then realized that it might be overlooked, so I'm reposting it as a new thread: Thanks. This one was caught a few days back and only recently merged. I guess it's time to roll out a new 2.6.23/powerpc I-pipe patch. http://download.gna.org/adeos/patches/v2.6/powerpc/adeos-ipipe-2.6.23-powerpc-DENX-2.0-03.patch -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Compile-time patch for mm_struct
Thank you very much. I hate to make extra work for you - is there a git repository for adeos-ipipe that I should be pulling from? Steve Philippe Gerum wrote: Philippe Gerum wrote: Steven A. Falco wrote: I needed the following patch to fix a forward reference (caught by gcc 4.1.1). I posted this as part of an earlier discussion, but I then realized that it might be overlooked, so I'm reposting it as a new thread: Thanks. This one was caught a few days back and only recently merged. I guess it's time to roll out a new 2.6.23/powerpc I-pipe patch. http://download.gna.org/adeos/patches/v2.6/powerpc/adeos-ipipe-2.6.23-powerpc-DENX-2.0-03.patch ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core