Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-27 Thread Benjie Chen
We got this bt after setting kernel dump stuff up. For some reason kgdb did not find debugging symbols, even though the kernel was compiled with -g. Hopefully this helps and I will go dig some more to find why we don't have a debug symbol. #0 0xc0674fae in doadump () #1 0xc067550a in boot () #2

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-27 Thread Benjie Chen
Sorry to waste your time. I discovered a similar PR with unp_gc and there may be a patch already. I will see if that works. Thanks, Benjie On 9/26/07, Benjie Chen [EMAIL PROTECTED] wrote: We got this bt after setting kernel dump stuff up. For some reason kgdb did not find debugging symbols,

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-26 Thread Benjie Chen
Well I spoke too soon. I increased the load even more and it crashed again. Same IP. So FYI, mpsafenet did not work, although it did allow me to stress the system for longer than I've ever done before. Perhaps that just reduced the concurrency enough. I will try kernel trace next. Thanks, Benjie

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-26 Thread Benjie Chen
You are right, they may not be the same. From first look it seems like they are similar based on the description of the problems -- system stable, then under load related to network, get panic after different time intervals. I just assumed that kernel is typically stable enough that this kind of

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-25 Thread Benjie Chen
Ivan and Kris, I will try to get a kernel trace -- it may not happen for awhile since I am not in the office and working remotely for awhile so it may not be easy to get a trace... but I will check. It looks like the problem reported by that link, and some of the links from there though...

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-25 Thread Kris Kennaway
Benjie Chen wrote: Ivan and Kris, I will try to get a kernel trace -- it may not happen for awhile since I am not in the office and working remotely for awhile so it may not be easy to get a trace... but I will check. It looks like the problem reported by that link, and some of the links from

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-25 Thread Ivan Voras
Kris Kennaway wrote: Does it really? i.e. did you compare the function names in detail and find that they match precisely, or do you just mean they are both panics of some description and I dunno what it all means? :) I ask because the linked trace does not involve a spinlock, which means it

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-25 Thread Kris Kennaway
Benjie Chen wrote: You are right, they may not be the same. From first look it seems like they are similar based on the description of the problems -- system stable, then under load related to network, get panic after different time intervals. I just assumed that kernel is typically stable

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-25 Thread Kris Kennaway
Ivan Voras wrote: Kris Kennaway wrote: Does it really? i.e. did you compare the function names in detail and find that they match precisely, or do you just mean they are both panics of some description and I dunno what it all means? :) I ask because the linked trace does not involve a

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-25 Thread Ivan Voras
On 25/09/2007, Kris Kennaway [EMAIL PROTECTED] wrote: Not in the sense of transmuting a sleep mutex into a spin mutex, no. sleep mutexes will spin when the lock holder is currently running, but this happens within the context of the mtx_lock_sleep function itself. Ok, thanks, this clears

Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Benjie Chen
Hi FreeBSD hackers and engineers, I am experiencing a kernel panic that comes on when my new PowerEdge 1950 FreeBSD 6.2 setup is under a certain stress load. I've emailed a few people on the list who have given me useful comments, some of which I am still following up. But I wanted to send a

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Borja Marcos
On 22 Sep 2007, at 00:26, Benjie Chen wrote: FreeBSD 6.2 on PowerEdge 1950, RAID1 setup with mfi driver (PERC5i). 4GB RAM. I am currently running i386, and not amd64, due to various reasons. Kernel panic is at 0xC066C731, which from nm shows it's in mtx_lock_spin c066c7b4 T

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Kris Kennaway
Borja Marcos wrote: On 22 Sep 2007, at 00:26, Benjie Chen wrote: FreeBSD 6.2 on PowerEdge 1950, RAID1 setup with mfi driver (PERC5i). 4GB RAM. I am currently running i386, and not amd64, due to various reasons. Kernel panic is at 0xC066C731, which from nm shows it's in mtx_lock_spin

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Kris Kennaway
Benjie Chen wrote: Hi FreeBSD hackers and engineers, I am experiencing a kernel panic that comes on when my new PowerEdge 1950 FreeBSD 6.2 setup is under a certain stress load. I've emailed a few people on the list who have given me useful comments, some of which I am still following up. But I

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Borja Marcos
On 24 Sep 2007, at 11:33, Kris Kennaway wrote: Borja Marcos wrote: I don't have the exact IP address involved, but we experienced consistent panics in two heavily loaded mail servers (same hardware models, Dell Powereedge) runnning Postfix and FreeBSD 6.2. Suspecting an issue with the IP

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Kris Kennaway
Borja Marcos wrote: On 24 Sep 2007, at 11:33, Kris Kennaway wrote: Borja Marcos wrote: I don't have the exact IP address involved, but we experienced consistent panics in two heavily loaded mail servers (same hardware models, Dell Powereedge) runnning Postfix and FreeBSD 6.2. Suspecting an

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Ivan Voras
Benjie Chen wrote: Kernel panic is at 0xC066C731, which from nm shows it's in mtx_lock_spin c066c7b4 T _mtx_lock_spin c066c85c T _mtx_unlock_sleep So this could mean that independent stress tests will not result in panic if there aren't enough concurrency to cause the problem. When you get

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Kris Kennaway
Ivan Voras wrote: Benjie Chen wrote: Kernel panic is at 0xC066C731, which from nm shows it's in mtx_lock_spin c066c7b4 T _mtx_lock_spin c066c85c T _mtx_unlock_sleep So this could mean that independent stress tests will not result in panic if there aren't enough concurrency to cause the

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Ivan Voras
On 24/09/2007, Kris Kennaway [EMAIL PROTECTED] wrote: Ivan Voras wrote: http://lists.freebsd.org/pipermail/freebsd-current/2007-September/076932.html Surely it cannot be since it involves a different function ;-) :) When all you have is a hammer...

Re: Kernel panic on PowerEdge 1950 under certain stress load

2007-09-24 Thread Ivan Voras
On 24/09/2007, Benjie Chen [EMAIL PROTECTED] wrote: Ivan and Kris, I will try to get a kernel trace -- it may not happen for awhile since I am not in the office and working remotely for awhile so it may not be easy to get a trace... but I will check. It's fairly easy: 1) add lines like the