Re: Oops in trace_hardirqs_on (powerpc)
On Wed, 2010-12-22 at 21:42 -0500, Steven Rostedt wrote: On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: Hi Steven, Did you've fixed this problem? The bug report is still marked as open. https://bugzilla.kernel.org/show_bug.cgi?id=16573 I just posted a patch to that BZ. I have it here below too. Could you see if it fixes you problem. I only fixed the one place that you reported, it may need more fixes (and in that case a macro to do the work). I hit the same bug on my ppc64 box, and have a fix for that, that I'll post to LKML tomorrow. Here's my official: Signed-off-by: Steven Rostedt rost...@goodmis.org -- Steve -- Steve diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index ed4aeb9..915cc03 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -879,7 +879,18 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_47x) */ andi. r10,r9,MSR_EE beq 1f + /* + * Since the ftrace irqsoff latency trace checks CALLER_ADDR1, + * which is the stack frame here, we need to force a stack frame + * in case we came from user space. + */ + stwur1,-32(r1) + mflrr0 + stw r0,4(r1) + stwur1,-32(r1) bl trace_hardirqs_on + lwz r1,0(r1) + lwz r1,0(r1) lwz r9,_MSR(r1) 1: #endif /* CONFIG_TRACE_IRQFLAGS */ ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Hi Steven, Steven Rostedt hat am Wed 22. Dec, 21:42 (-0500) geschrieben: On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: Did you've fixed this problem? The bug report is still marked as open. https://bugzilla.kernel.org/show_bug.cgi?id=16573 I just posted a patch to that BZ. I have it here below too. Could you see if it fixes you problem. I only fixed the one place that you reported, it may need more fixes (and in that case a macro to do the work). I hit the same bug on my ppc64 box, and have a fix for that, that I'll post to LKML tomorrow. -- Steve diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index ed4aeb9..915cc03 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -879,7 +879,18 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_47x) */ andi. r10,r9,MSR_EE beq 1f + /* + * Since the ftrace irqsoff latency trace checks CALLER_ADDR1, + * which is the stack frame here, we need to force a stack frame + * in case we came from user space. + */ + stwur1,-32(r1) + mflrr0 + stw r0,4(r1) + stwur1,-32(r1) bl trace_hardirqs_on + lwz r1,0(r1) + lwz r1,0(r1) lwz r9,_MSR(r1) 1: #endif /* CONFIG_TRACE_IRQFLAGS */ This patch eliminates the oopses. Bye, Jörg. -- Der Klügere gibt so lange nach bis er der Dumme ist. signature.asc Description: Digital signature http://en.wikipedia.org/wiki/OpenPGP ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: Hi Steven, Did you've fixed this problem? The bug report is still marked as open. https://bugzilla.kernel.org/show_bug.cgi?id=16573 I just posted a patch to that BZ. I have it here below too. Could you see if it fixes you problem. I only fixed the one place that you reported, it may need more fixes (and in that case a macro to do the work). I hit the same bug on my ppc64 box, and have a fix for that, that I'll post to LKML tomorrow. -- Steve diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index ed4aeb9..915cc03 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -879,7 +879,18 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_47x) */ andi. r10,r9,MSR_EE beq 1f + /* +* Since the ftrace irqsoff latency trace checks CALLER_ADDR1, +* which is the stack frame here, we need to force a stack frame +* in case we came from user space. +*/ + stwur1,-32(r1) + mflrr0 + stw r0,4(r1) + stwur1,-32(r1) bl trace_hardirqs_on + lwz r1,0(r1) + lwz r1,0(r1) lwz r9,_MSR(r1) 1: #endif /* CONFIG_TRACE_IRQFLAGS */ ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: Hi Steven, Steven Rostedt hat am Mon 27. Sep, 21:58 (-0400) geschrieben: On Mon, 2010-09-27 at 14:50 +0200, Jörg Sommer wrote: Hello Steven, Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: Sorry for the late reply, but I was on vacation when you sent this, and I missed it while going through email. Do you still have this issue? No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem vanished, as expected. The problem is, that in some cases the stack is only two frames deep, which causes the macro CALLER_ADDR1 makes an invalid access. Someone told me, there a workaround for the problem on i386, too. % sed -n 2p arch/x86/lib/thunk_32.S * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) Yes, I remember that problem. When I get back from Tokyo, I'll tried to remember to fix it. Did you've fixed this problem? The bug report is still marked as open. https://bugzilla.kernel.org/show_bug.cgi?id=16573 Ah, this email got lost in the hundreds I had when I got back from Tokyo, sorry about that again :-( Anyway, it looks like this only affects 32 bit PPC as I can't reproduce it with my 64 bit one. And also, unfortunately, my 32bit ppc got taken from me by my kids, so I can't test it on that either. I'll look to see if I can write up a patch. Perhaps you could test it for me. Thanks, -- Steve ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Mon, 2010-12-20 at 15:43 -0500, Steven Rostedt wrote: Anyway, it looks like this only affects 32 bit PPC as I can't reproduce it with my 64 bit one. And also, unfortunately, my 32bit ppc got taken from me by my kids, so I can't test it on that either. Spoke too soon, I just triggered it on 64bit. I'll look into it. Thanks! -- Steve ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Hi Steven, Steven Rostedt hat am Mon 27. Sep, 21:58 (-0400) geschrieben: On Mon, 2010-09-27 at 14:50 +0200, Jörg Sommer wrote: Hello Steven, Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: Sorry for the late reply, but I was on vacation when you sent this, and I missed it while going through email. Do you still have this issue? No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem vanished, as expected. The problem is, that in some cases the stack is only two frames deep, which causes the macro CALLER_ADDR1 makes an invalid access. Someone told me, there a workaround for the problem on i386, too. % sed -n 2p arch/x86/lib/thunk_32.S * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) Yes, I remember that problem. When I get back from Tokyo, I'll tried to remember to fix it. Did you've fixed this problem? The bug report is still marked as open. https://bugzilla.kernel.org/show_bug.cgi?id=16573 Regards, Jörg. -- Begebenheit aus dem wahren Leben: Mediziner: ICEs sind die weißen Züge. Mathematiker: Das ist falsch. Jeder ICE ist zwar weiß, aber nicht alle weißen Züge sind ICEs. signature.asc Description: Digital signature http://en.wikipedia.org/wiki/OpenPGP ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Hello Steven, Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: Sorry for the late reply, but I was on vacation when you sent this, and I missed it while going through email. Do you still have this issue? No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem vanished, as expected. The problem is, that in some cases the stack is only two frames deep, which causes the macro CALLER_ADDR1 makes an invalid access. Someone told me, there a workaround for the problem on i386, too. % sed -n 2p arch/x86/lib/thunk_32.S * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) Bye, Jörg. -- Angenehme Worte sind nie wahr, wahre Worte sind nie angenehm. signature.asc Description: Digital signature http://en.wikipedia.org/wiki/OpenPGP ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Mon, 2010-09-27 at 14:50 +0200, Jörg Sommer wrote: Hello Steven, Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: Sorry for the late reply, but I was on vacation when you sent this, and I missed it while going through email. Do you still have this issue? No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem vanished, as expected. The problem is, that in some cases the stack is only two frames deep, which causes the macro CALLER_ADDR1 makes an invalid access. Someone told me, there a workaround for the problem on i386, too. % sed -n 2p arch/x86/lib/thunk_32.S * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) Yes, I remember that problem. When I get back from Tokyo, I'll tried to remember to fix it. Thanks! -- Steve ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Sorry for the late reply, but I was on vacation when you sent this, and I missed it while going through email. Do you still have this issue? -- Steve On Sat, 2010-08-07 at 01:31 +0200, Jörg Sommer wrote: Hi, I've built my 2.6.35 with tracing support and now, I'm getting continuously oops'. It seems to happen on high process activity. [ 52.336371] device eth0 entered promiscuous mode [ 52.347616] device eth0 left promiscuous mode [ 55.240663] Unable to handle kernel paging request for data at address 0xbfaf4a24 [ 55.248289] Faulting instruction address: 0xc00aad98 [ 55.255562] Oops: Kernel access of bad area, sig: 11 [#1] [ 55.262588] PowerMac [ 55.269606] last sysfs file: /sys/devices/pci:00/:00:10.0/graphics/fb0/radeonbl0/brightness [ 55.277111] Modules linked in: fuse snd_powermac option usb_wwan usbserial ecb b43 snd_aoa_i2sbus snd_pcm_oss [ 55.302368] NIP: c00aad98 LR: c001771c CTR: c003dba0 [ 55.310738] REGS: e3211e70 TRAP: 0300 Not tainted (2.6.35) [ 55.319122] MSR: 1032 ME,IR,DR CR: 22f88f42 XER: 2000 [ 55.327650] DAR: bfaf4a24, DSISR: 4000 [ 55.335954] TASK = e3245bc0[1929] 'sh' THREAD: e321 [ 55.336144] GPR00: e3211f20 e3245bc0 e3245bc0 c000b944 003a1040 [ 55.344859] GPR08: bfaf4a20 c05e c0614d18 c061 10033368 10018520 10007c2c [ 55.353723] GPR16: 10007c30 bfecaa10 101d8304 10019c28 bfecbfab [ 55.362438] GPR24: bfecaa08 10019c58 06d1 c063be80 bfeca9a0 0ffebff4 e3211f20 [ 55.378913] NIP [c00aad98] trace_hardirqs_on+0x5c/0x124 [ 55.386856] LR [c001771c] restore+0x10/0x6c [ 55.394527] Call Trace: [ 55.401878] [e3211f20] [10019c58] 0x10019c58 (unreliable) [ 55.409437] [e3211f40] [c001771c] restore+0x10/0x6c [ 55.417065] --- Exception: c00 at 0xff23c88 [ 55.417071] LR = 0xff23c54 [ 55.432267] Instruction dump: [ 55.439808] 800a005c 70090002 418200c8 7ca6 70008000 408200bc 3d20c05e 838a0058 [ 55.447730] 81096f98 2f88 811f 8108 83680004 41be009c 816b4d18 90096f98 [ 55.455722] ---[ end trace 547f1189532873f7 ]--- [ 390.022834] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 507.793120] lo: Disabled Privacy Extensions [ 518.228969] eth0: no IPv6 routers present [ 737.593898] Unable to handle kernel paging request for data at address 0x0004 [ 737.593927] Faulting instruction address: 0xc00aad98 [ 737.593957] Oops: Kernel access of bad area, sig: 11 [#2] [ 737.593967] PowerMac [ 737.593976] last sysfs file: /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor [ 737.593992] Modules linked in: ppp_async crc_ccitt ipv6 ppp_generic slhc fuse snd_powermac option usb_wwan usb [ 737.594132] NIP: c00aad98 LR: c001771c CTR: c003dba0 [ 737.594148] REGS: e685de70 TRAP: 0300 Tainted: G D (2.6.35) [ 737.594159] MSR: 1032 ME,IR,DR CR: 2442 XER: 2000 [ 737.594187] DAR: 0004, DSISR: 4000 [ 737.594198] TASK = e30b3780[3322] 'zsh-beta' THREAD: e685c000 [ 737.594208] GPR00: e685df20 e30b3780 e30b3780 c000b944 003e5f00 [ 737.594240] GPR08: c05e c0614d18 c061 100b4ee8 10092dec [ 737.594271] GPR16: 100bb400 100916fc bfbda1b0 bfbda4ec [ 737.594303] GPR24: 100b 100bae50 0cea c063be80 bfbd9e60 0fe64ff4 e685df20 [ 737.594362] NIP [c00aad98] trace_hardirqs_on+0x5c/0x124 [ 737.594379] LR [c001771c] restore+0x10/0x6c [ 737.594388] Call Trace: [ 737.594402] [e685df20] [100bae50] 0x100bae50 (unreliable) [ 737.594421] [e685df40] [c001771c] restore+0x10/0x6c [ 737.594432] Instruction dump: [ 737.594442] 800a005c 70090002 418200c8 7ca6 70008000 408200bc 3d20c05e 838a0058 [ 737.594473] 81096f98 2f88 811f 8108 83680004 41be009c 816b4d18 90096f98 [ 737.594514] ---[ end trace 547f1189532873f8 ]--- [ 737.919108] Unable to handle kernel paging request for data at address 0x0003 [ 737.919137] Faulting instruction address: 0xc00aad98 [ 737.919168] Oops: Kernel access of bad area, sig: 11 [#3] [ 737.919179] PowerMac [ 737.919187] last sysfs file: /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor [ 737.919203] Modules linked in: ppp_async crc_ccitt ipv6 ppp_generic slhc fuse snd_powermac option usb_wwan usb [ 737.919342] NIP: c00aad98 LR: c001771c CTR: [ 737.919358] REGS: e6d15e70 TRAP: 0300 Tainted: G D (2.6.35) [ 737.919369] MSR: 1032 ME,IR,DR CR: 2442 XER: [ 737.919397] DAR: 0003, DSISR: 4000 [ 737.919409] TASK = e30b3780[3350] 'zsh-beta' THREAD: e6d14000 [ 737.919419] GPR00: e6d15f20 e30b3780 e30b3780 c000b944 0065df00 0008 [ 737.919451] GPR08: c05e c0614d18 c061 100b4ee8 100ad1e8
