Re: Strange crashes started this morning
On Fri, Jun 22, 2007 at 11:27:11PM -0400, Alex Feldman wrote: Hi Andrew You crash dump doesn't show that it crashed on san driver. I'm saying that this is not the problem with san driver but it doesn't show any driver related function in crash trace. I do not see that either. However, I am not familier with the internals of the OpenBSD kernel. Theo is, and he seems to think it is a san issue. At this point I trust his judgment above yours. My suggestion would be to provide the documentation that the OpenBSD team is looking for so that they can prove one way or another where the problems are and improve the code for everyone. For both Andrew and Richard: 1. If you can send me the crash trace that includes san driver function that will be helpful. I expect that it would, unfortunatly, I cannot reproduce this problem on command. It only happened the one day so far and I have no idea what caused it. How about, while waiting for more information on this problem, you see if you can do anything about a problem I can repeat. It causes me no end of trouble because it makes both routers DDB any time I soft boot them. That means I can't upgrade the version of OpenBSD on them remotely. This I attribute to the san stuff because it doesn't happen in any of the other machines I am running OpenBSD on. You may notice that the trace for this one also doesn't reference any san driver calls. You can see it in bug number 5404: http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yesnumbers=5404 In bug 4484, someone else seems to have had similar issues: http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yesnumbers=4484 Who knows, getting the OpenBSD developers the documentation they need so they can fix that issue will coincidently fix the one I am complaining about now. 2. Can you send me the configuration for ppp/Wanpipe and details instruction how to get this crash; I'll try to resolve this issue. Here is the configuration on the interfaces that seemed to cause the issue this last time. They are they only lines I have that are PROTO=ppp, the rest are HDLC (PROTO=cisco). $ sudo sanconfig san2 ALEX2 Hardware configuration for san2: AFT-A102 : SLOT=8 : BUS=0 : IRQ=10 : CPU=A : PORT=PRI Interface configuration for san2: MEDIA=T1 LCODE=B8ZS FRAME=ESF TECLOCK=Normal LBO=0db ACTIVE_CH=all PROTO=ppp $ sudo sanconfig san3 ALEX2 Hardware configuration for san3: AFT-A102 : SLOT=8 : BUS=0 : IRQ=10 : CPU=B : PORT=PRI Interface configuration for san3: MEDIA=T1 LCODE=B8ZS FRAME=ESF TECLOCK=Normal LBO=0db ACTIVE_CH=all PROTO=ppp For the other, I will just quote what I wrote before. From: andrew fresh [mailto:[EMAIL PROTECTED] There are two resons I believe it is the Sangoma driver causing the problem. The first is the message from Theo that you can read in the archives here: http://marc.info/?l=openbsd-miscm=118246162917905w=2 where he said I suggest you call [Sangoma]. The second being the logs. They are in the messages linked above, but just before the router locked up there were san messages in /var/log/messages and on the console there is san2: LCP keepalive timeout. And this: I am not sure what triggers this problem and it has not happened again since the times mentioned in that email so it is fairly difficult to debug. l8rZ, -- andrew - ICQ# 253198 - Jabber: [EMAIL PROTECTED] BOFH excuse of the day: Pentium FDIV bug
Re: Strange crashes started this morning
Hello Andrew, I'm sorry for the delay. I don't have always time to got through mailing list. It is not so clear that the crash related to Sangoma driver. I would like to see the crash dump at that moment. This will help me to resolve the issue. Alex -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of andrew fresh Sent: Thursday, June 21, 2007 7:27 PM To: misc@openbsd.org Subject: Re: Strange crashes started this morning On Thu, Jun 21, 2007 at 03:29:04PM -0600, Theo de Raadt wrote: Sangoma has made it pretty clear (by ignoring problem reports from our developers) that they don't care at all. I suggest you call them, and add your voice there. I certainly will. Unfortunately I think these cards showed up on the same day that the announcement about accoom.kd85.com showed up in my mailbox. Is there anyone in particular I should ask to speak with there or just anyone who answers the phone? Otherwise, I suggest that everyone running Sangoma projects learn from this experience... I certainly would recommend a different card. I am trying to replace some of these with an ethernet connection but getting anything other than T1s has been a problem here. When I do get this fast ethernet line, does anyone want to trade 5 dual port san(4) cards for 3 dual port art(4)? :-) Otherwise I will have to try to get a budget approved to just replace them. l8rZ, -- andrew - ICQ# 253198 - Jabber: [EMAIL PROTECTED] At the source of every error which is blamed on the computer, you will find at least two human errors, including the error of blaming it on the computer.
Re: Strange crashes started this morning
On Fri, Jun 22, 2007 at 04:45:34PM -0400, Alex Feldman wrote: Hello Andrew, I'm sorry for the delay. I don't have always time to got through mailing list. It is not so clear that the crash related to Sangoma driver. I would like to see the crash dump at that moment. This will help me to resolve the issue. I included the trace and other information in my original message to the list. You can see it in the archives here: http://marc.info/?l=openbsd-miscm=118245939832197w=2 I also submitted that information to [EMAIL PROTECTED] from [EMAIL PROTECTED] There are two resons I believe it is the Sangoma driver causing the problem. The first is the message from Theo that you can read in the archives here: http://marc.info/?l=openbsd-miscm=118246162917905w=2 where he said I suggest you call [Sangoma]. The second being the logs. They are in the messages linked above, but just before the router locked up there were san messages in /var/log/messages and on the console there is san2: LCP keepalive timeout. Is there some additional information that you need that I can provide? I believe everything is in that first message linked above. I am not sure what triggers this problem and it has not happened again since the times mentioned in that email so it is fairly difficult to debug. I worry that it will happen when I am not available to restart it and so would like to get it resolved. l8rZ, -- andrew - ICQ# 253198 - Jabber: [EMAIL PROTECTED] A printer consists of three main parts: the case, the jammed paper tray and the blinking red light.
Re: Strange crashes started this morning
Hi Andrew You crash dump doesn't show that it crashed on san driver. I'm saying that this is not the problem with san driver but it doesn't show any driver related function in crash trace. For both Andrew and Richard: 1. If you can send me the crash trace that includes san driver function that will be helpful. 2. Can you send me the configuration for ppp/Wanpipe and details instruction how to get this crash; I'll try to resolve this issue. Alex -Original Message- From: andrew fresh [mailto:[EMAIL PROTECTED] Sent: Friday, June 22, 2007 5:09 PM To: Alex Feldman Subject: Re: Strange crashes started this morning On Fri, Jun 22, 2007 at 04:45:34PM -0400, Alex Feldman wrote: Hello Andrew, I'm sorry for the delay. I don't have always time to got through mailing list. It is not so clear that the crash related to Sangoma driver. I would like to see the crash dump at that moment. This will help me to resolve the issue. I included the trace and other information in my original message to the list. You can see it in the archives here: http://marc.info/?l=openbsd-miscm=118245939832197w=2 I also submitted that information to [EMAIL PROTECTED] from [EMAIL PROTECTED] There are two resons I believe it is the Sangoma driver causing the problem. The first is the message from Theo that you can read in the archives here: http://marc.info/?l=openbsd-miscm=118246162917905w=2 where he said I suggest you call [Sangoma]. The second being the logs. They are in the messages linked above, but just before the router locked up there were san messages in /var/log/messages and on the console there is san2: LCP keepalive timeout. Is there some additional information that you need that I can provide? I believe everything is in that first message linked above. I am not sure what triggers this problem and it has not happened again since the times mentioned in that email so it is fairly difficult to debug. I worry that it will happen when I am not available to restart it and so would like to get it resolved. l8rZ, -- andrew - ICQ# 253198 - Jabber: [EMAIL PROTECTED] A printer consists of three main parts: the case, the jammed paper tray and the blinking red light.
Re: Strange crashes started this morning
I see san2: LCP keepalive timeout output to the console and the below in /var/log/messages right around when it locked up but the traces don't seem to have anything to do with the network. Jun 21 06:05:05 rrlhcrtr0200 /bsd: san3: T1 YELLOW ON Jun 21 06:05:05 rrlhcrtr0200 /bsd: san3: T1 disconnected! Jun 21 06:05:05 rrlhcrtr0200 /bsd: san3: Link connecting... Jun 21 06:40:56 rrlhcrtr0200 syslogd: start Jun 21 07:15:44 rrlhcrtr0200 /bsd: san3: T1 LB activation code received. Jun 21 07:15:44 rrlhcrtr0200 /bsd: san3: Unknown signal (15). Jun 21 07:15:59 rrlhcrtr0200 /bsd: san3: T1 LB deactivation code received. Jun 21 07:16:04 rrlhcrtr0200 /bsd: san3: T1 LB deactivation code received. Jun 21 07:16:08 rrlhcrtr0200 /bsd: san3: Unknown signal (09). Jun 21 07:16:08 rrlhcrtr0200 /bsd: san3: Unknown signal (15). Jun 21 07:16:15 rrlhcrtr0200 /bsd: san2: T1 LB activation code received. Jun 21 07:16:15 rrlhcrtr0200 /bsd: san2: Unknown signal (15). Jun 21 07:16:39 rrlhcrtr0200 /bsd: san2: T1 LB deactivation code received. Jun 21 08:08:49 rrlhcrtr0200 syslogd: start Jun 21 09:26:53 rrlhcrtr0200 /bsd: san2: T1 AIS ON Jun 21 09:26:53 rrlhcrtr0200 /bsd: san2: T1 disconnected! Jun 21 09:26:53 rrlhcrtr0200 /bsd: san2: Link connecting... Jun 21 09:26:54 rrlhcrtr0200 /bsd: san2: T1 RED ON Jun 21 09:30:51 rrlhcrtr0200 syslogd: start Jun 21 09:39:51 rrlhcrtr0200 /bsd: san2: T1 AIS ON Jun 21 09:39:51 rrlhcrtr0200 /bsd: san2: T1 disconnected! Jun 21 09:39:51 rrlhcrtr0200 /bsd: san2: Link connecting... Jun 21 09:39:52 rrlhcrtr0200 /bsd: san2: T1 RED ON Jun 21 09:50:21 rrlhcrtr0200 syslogd: start Jun 21 09:50:21 rrlhcrtr0200 syslogd: start Sangoma has made it pretty clear (by ignoring problem reports from our developers) that they don't care at all. I suggest you call them, and add your voice there. Otherwise, I suggest that everyone running Sangoma projects learn from this experience...
Re: Strange crashes started this morning
On 6/21/07, andrew fresh [EMAIL PROTECTED] wrote: I have several routers that have been running great for many months. (even better since I upgraded to 4.1 on them oround May 4th) OpenBSD 4.1-stable (GENERIC.MP) #0: Fri May 4 21:56:51 MST 2007 This morning, one of them went down and nagios paged me. Getting to work, I just thought it was odd, looked at the trace and restarted it and went home. About half an hour later, it happened again. I again what happens if you push c and enter?
Re: Strange crashes started this morning
On Thu, Jun 21, 2007 at 03:29:04PM -0600, Theo de Raadt wrote: Sangoma has made it pretty clear (by ignoring problem reports from our developers) that they don't care at all. I suggest you call them, and add your voice there. I certainly will. Unfortunately I think these cards showed up on the same day that the announcement about accoom.kd85.com showed up in my mailbox. Is there anyone in particular I should ask to speak with there or just anyone who answers the phone? Otherwise, I suggest that everyone running Sangoma projects learn from this experience... I certainly would recommend a different card. I am trying to replace some of these with an ethernet connection but getting anything other than T1s has been a problem here. When I do get this fast ethernet line, does anyone want to trade 5 dual port san(4) cards for 3 dual port art(4)? :-) Otherwise I will have to try to get a budget approved to just replace them. l8rZ, -- andrew - ICQ# 253198 - Jabber: [EMAIL PROTECTED] At the source of every error which is blamed on the computer, you will find at least two human errors, including the error of blaming it on the computer.
