Re: checking stopevent 2!
On Sat, 15 Nov 2003, Andy Farkas wrote:
These messages spew onto my console and into syslogd once every second:
Heh. Sounds like your box is having a really bad day, we'll see if we
can't get it fixed up over the next couple of weeks as things settle out
:-).
I think John has this one in his sights already, we talked about the
sigacts locking during the release engineering telecon yesterday (CC'd
gratuitously in this message). Basically, every place where a debugger
can stop a process there's a call to STOPEVENT(), which may sleep
indefinitely waiting to be restarted by the process performing the
debugging. The problem appears to be that recent changes pushed a
STOPEVENT() call inside code holding a mutex, so the locking of that piece
of code needs to be changed slightly to make sure that doesn't happen. IT
would probably be useful if you could drop to DDB and generate a trace for
the event.
Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
[EMAIL PROTECTED] Network Associates Laboratories
...
Nov 15 16:05:44 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:44 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/kern_condvar.c:289
Nov 15 16:05:44 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:44 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:44 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4663aa8) locked @ /hummer/src-current/src/sys/kern/kern_synch.c:293
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4663aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4663aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/kern_condvar.c:289
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:46 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/kern_condvar.c:289
Nov 15 16:05:46 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:46 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
...
This is latest -current (cvsup'd a few hours ago)
--
:{ [EMAIL PROTECTED]
Andy Farkas
System Administrator
Speednet Communications
http://www.speednet.com.au/
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: checking stopevent 2!
On Sat, Nov 15, 2003 at 09:38:37AM -0500, Robert Watson wrote:
On Sat, 15 Nov 2003, Andy Farkas wrote:
would probably be useful if you could drop to DDB and generate a trace for
the event.
I've done that, in this email message:
http://docs.freebsd.org/cgi/getmsg.cgi?fetch=2157067+0+current/freebsd-current
...
Nov 15 16:05:44 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:44 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/kern_condvar.c:289
Nov 15 16:05:44 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:44 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:44 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4663aa8) locked @ /hummer/src-current/src/sys/kern/kern_synch.c:293
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4663aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4663aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:45 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/kern_condvar.c:289
Nov 15 16:05:45 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:46 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/kern_condvar.c:289
Nov 15 16:05:46 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
Nov 15 16:05:46 kern.crit hummer kernel: checking stopevent 2 with the following
non-sleepable locks held:
Nov 15 16:05:46 kern.crit hummer kernel: exclusive sleep mutex sigacts r = 0
(0xc4656aa8) locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
...
This is latest -current (cvsup'd a few hours ago)
--
:{ [EMAIL PROTECTED]
Andy Farkas
System Administrator
Speednet Communications
http://www.speednet.com.au/
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Cosmin Stroe
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: checking stopevent 2!
On Sat, Nov 15, 2003 at 09:38:37AM -0500, Robert Watson wrote: On Sat, 15 Nov 2003, Andy Farkas wrote: These messages spew onto my console and into syslogd once every second: I'm seeing the same on a recently upgraded dual cpu machine. Also when I run the reboot command the errors get mixed on the console as if the process writing to the console wasn't locking the device. A verbose dmesg can be found at http://xbsd.net/~esbjerg/enzo.verb.dmesg and a kernel conf http://xbsd.net/~esbjerg/ENZO I might add that the recent changes has totally hosed the USB support... regards Sven Esbjerg -- http://www.usenet.dk/netikette - på forhånd tak. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: checking stopevent 2!
On Sat, 15 Nov 2003, Sven Esbjerg wrote: On Sat, Nov 15, 2003 at 09:38:37AM -0500, Robert Watson wrote: On Sat, 15 Nov 2003, Andy Farkas wrote: These messages spew onto my console and into syslogd once every second: I'm seeing the same on a recently upgraded dual cpu machine. Also when I run the reboot command the errors get mixed on the console as if the process writing to the console wasn't locking the device. A verbose dmesg can be found at http://xbsd.net/~esbjerg/enzo.verb.dmesg and a kernel conf http://xbsd.net/~esbjerg/ENZO I might add that the recent changes has totally hosed the USB support... Could you be more specific about the USB problems? Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Network Associates Laboratories ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: checking stopevent 2!
On 15 Nov, Robert Watson wrote: On Sat, 15 Nov 2003, Andy Farkas wrote: These messages spew onto my console and into syslogd once every second: Heh. Sounds like your box is having a really bad day, we'll see if we can't get it fixed up over the next couple of weeks as things settle out :-). Mine is worse. If I try to boot it multiuser, it pegs its serial console with these messages. It seems to bring up the network to the point where it can be pinged, but the ping latency averages about 200 ms, and even after a half an hour it still hadn't gotten sshd started. I can't drop back to the old kernel because I just did an installworld with the new version of statfs. I hope the fix isn't too extensive, since I'll probably be typing it in by hand in single user mode ... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: checking stopevent 2!
In message: [EMAIL PROTECTED] Don Lewis [EMAIL PROTECTED] writes: : I hope the fix isn't too extensive, since I'll probably be typing it in : by hand in single user mode ... You could do what I did: remove the two lines that jhb added. You'll get no more warnings. Sure, the problems are still there, but jhb likely will fix them, at which time you'll be able to run stock proc.h again. Warner ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: checking stopevent 2!
On Sat, 15 Nov 2003, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Don Lewis [EMAIL PROTECTED] writes: : I hope the fix isn't too extensive, since I'll probably be typing it in : by hand in single user mode ... You could do what I did: remove the two lines that jhb added. You'll get no more warnings. Sure, the problems are still there, but jhb likely will fix them, at which time you'll be able to run stock proc.h again. Maybe we should just go ahead and commit that change until John has a chance to merge the actual fix? I talked with him a bit about it on Friday, but I'm not sure whether he's working on it this weekend, or plans to address it on Monday. He may not have realized the problem was quite so debilitating as the problem apparently turns out to be... Robert N M Watson FreeBSD Core Team, TrustedBSD Projects [EMAIL PROTECTED] Network Associates Laboratories ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: checking stopevent 2!
Robert Watson wrote:
On Sat, 15 Nov 2003, Andy Farkas wrote:
These messages spew onto my console and into syslogd once every second:
Heh. Sounds like your box is having a really bad day, we'll see if we
can't get it fixed up over the next couple of weeks as things settle out
:-).
As you've probably noticed, this problem is pretty wide-spread. A fix
better get in before 5.2-release please :)
I've managed to stop the message flooding by not running ntpd.
IT would probably be useful if you could drop to DDB and generate a
trace for the event.
Do you still want me to do this? I thought the messages themselves were
supposed to useful:
locked @ /hummer/src-current/src/sys/kern/kern_condvar.c:289
locked @ /hummer/src-current/src/sys/kern/kern_synch.c:293
locked @ /hummer/src-current/src/sys/kern/subr_trap.c:260
# ident /hummer/src-current/src/sys/kern/kern_condvar.c
/hummer/src-current/src/sys/kern/kern_condvar.c:
$FreeBSD: src/sys/kern/kern_condvar.c,v 1.44 2003/11/09 09:17:24 tanimura Exp $
# ident /hummer/src-current/src/sys/kern/kern_synch.c
/hummer/src-current/src/sys/kern/kern_synch.c:
$FreeBSD: src/sys/kern/kern_synch.c,v 1.237 2003/10/29 15:23:09 bde Exp $
# ident /hummer/src-current/src/sys/kern/subr_trap.c
/hummer/src-current/src/sys/kern/subr_trap.c:
$FreeBSD: src/sys/kern/subr_trap.c,v 1.261 2003/09/05 22:15:26 peter Exp $
--
:{ [EMAIL PROTECTED]
Andy Farkas
System Administrator
Speednet Communications
http://www.speednet.com.au/
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
