Re: [asterisk-users] Strange network issue
On Thu, Jul 28, 2011 at 4:46 AM, Duncan Turnbull wrote: > On 28/07/2011, at 8:41 PM, Paul Hayes wrote: > > > On 28/07/11 02:58, Mike Diehl wrote: > >> > >> Any ideas? > >> > >> Mike. > > > > I'd go on site if possible and see what actually happens at 19:00. Set > up a wireshark trace capturing all traffic through their router. > > > > -- > I am picking a cleaner plugging a powerful vacuum cleaner in ;-) > > That's what I mentioned earlier, but thinking about it they must have a German cleaning service to get such precise vacuum timing. -M -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
On 28/07/2011, at 8:41 PM, Paul Hayes wrote: > On 28/07/11 02:58, Mike Diehl wrote: >> >> Any ideas? >> >> Mike. > > I'd go on site if possible and see what actually happens at 19:00. Set up a > wireshark trace capturing all traffic through their router. > > -- I am picking a cleaner plugging a powerful vacuum cleaner in ;-) -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
On 28/07/11 02:58, Mike Diehl wrote: Any ideas? Mike. I'd go on site if possible and see what actually happens at 19:00. Set up a wireshark trace capturing all traffic through their router. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
Well, it gets stranger... As you can see from the syslog extract below, the phone registers every 5 minutes quite reliably, until 19:02. From this time forward, the phone is unable to register. No one touched the phone and I didn't change my RT Asterisk configuration. Any ideas? Mike. === Jul 27 18:29:17 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][000][962522A][0101062C] 000B821CA9B6-1:REGISTERED for 300 seconds;re-REGISTER in 280 seconds Jul 27 18:33:57 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][000][962522A][0101062C] 000B821CA9B6-1:REGISTERED for 300 seconds;re-REGISTER in 280 seconds Jul 27 18:38:37 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][000][962522A][0101062C] 000B821CA9B6-1:REGISTERED for 300 seconds;re-REGISTER in 280 seconds Jul 27 18:43:17 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][000][962522A][0101062C] 000B821CA9B6-1:REGISTERED for 300 seconds;re-REGISTER in 280 seconds Jul 27 18:47:57 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][000][962522A][0101062C] 000B821CA9B6-1:REGISTERED for 300 seconds;re-REGISTER in 280 seconds Jul 27 18:52:38 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][000][962522A][0101062C] 000B821CA9B6-1:REGISTERED for 300 seconds;re-REGISTER in 280 seconds Jul 27 18:57:18 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][000][962522A][0101062C] 000B821CA9B6-1:REGISTERED for 300 seconds;re-REGISTER in 280 seconds Jul 27 19:02:09 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][501][962522A][0101062C] 000B821CA9B6-1 SIP registration failed. Retrying in 20 seconds. Server: 209.250.31.96 Jul 27 19:02:41 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][501][962522A][0101062C] 000B821CA9B6-1 SIP registration failed. Retrying in 20 seconds. Server: 209.250.31.96 Jul 27 19:03:12 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][501][962522A][0101062C] 000B821CA9B6-1 SIP registration failed. Retrying in 20 seconds. Server: 209.250.31.96 Jul 27 19:03:44 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][501][962522A][0101062C] 000B821CA9B6-1 SIP registration failed. Retrying in 20 seconds. Server: 209.250.31.96 Jul 27 19:04:16 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][501][962522A][0101062C] 000B821CA9B6-1 SIP registration failed. Retrying in 20 seconds. Server: 209.250.31.96 Jul 27 19:04:47 c-107-4-61-188.hsd1.nm.comcast.net GS_LOG: [00:0B:82:1C:A9:B6][501][962522A][0101062C] 000B821CA9B6-1 SIP registration failed. Retrying in 20 seconds. Server: 209.250.31.96 === On Friday 22 July 2011 2:38:15 am Mike Diehl wrote: > On Friday 22 July 2011 1:42:33 am Ishfaq Malik wrote: > > On Thu, 2011-07-21 at 17:13 -0600, Mike Diehl wrote: > > > Hi all, > > > > > > I've got a strange problem with a customer's phones. > > > > > > They've got a bunch of Grandstreams that seem to be rock solid... until > > > 7:00pm. At 7:00, some of the phones become unavailable, and stay down. > > > Call quality is solid almost all the time. But right at 7:00, things > > > go bad. Only some of the phone lines go down and they stay down until > > > the phone is rebooted. > > > > > > I'm not even sure what to look for when I go to the site. Any ideas? > > > > Are the phones running on the same connection as the computers/servers? > > If so, are they doing any scheduled backups? > > > > Ish > > No, the phones are in a different city than the servers and connected via > the Internet. > > The users probably aren't sophisticated enough to be running a scheduled > backup... -- Take care and have fun, Mike Diehl. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
On Fri, 2011-07-22 at 10:58 -0700, Dave Platt wrote: > > They've got a bunch of Grandstreams that seem to be rock solid... until > > 7:00pm. At 7:00, some of the phones become unavailable, and stay down. > > Call > > quality is solid almost all the time. But right at 7:00, things go bad. > > Only > > some of the phone lines go down and they stay down until the phone is > > rebooted. > > > > I'm not even sure what to look for when I go to the site. Any ideas? > > I'd look to see if there are any electrical circuits (lights, > fans, etc.) which are on a timer of some sort, and are automatically > powered off at 7 PM. > > If somebody mistakenly plugged a piece of network kit into such a > circuit, it would lose power at that time, and your network might > end up being partitioned, or routing (switch or IP-level) might > change abruptly. > Hi, Even if there is no equipment you own controlled by a timer, you still can suffer from it. Some power companies have different rates for power you use during daytime or at night. So even if _you_ don't have equipment on a timer, your neighbours might have. Something like electrical boilers or so, or other "heavy equipment". Switching them on/off can cause huge spikes on the electrical wires. A couple of neigbours at work have their own micro-power-generators. About one in ten times, when they start delevering power to the grid, all of our test-systems go down. Only the systems behind the re-generated UPS (that removes spikes from the powerlines) are protected against them. So nasty litte spikes are harder to detect/tracedown than a full blackout. hw -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
> They've got a bunch of Grandstreams that seem to be rock solid... until > 7:00pm. At 7:00, some of the phones become unavailable, and stay down. Call > quality is solid almost all the time. But right at 7:00, things go bad. > Only > some of the phone lines go down and they stay down until the phone is > rebooted. > > I'm not even sure what to look for when I go to the site. Any ideas? I'd look to see if there are any electrical circuits (lights, fans, etc.) which are on a timer of some sort, and are automatically powered off at 7 PM. If somebody mistakenly plugged a piece of network kit into such a circuit, it would lose power at that time, and your network might end up being partitioned, or routing (switch or IP-level) might change abruptly. If (for example) the phones were being DHCP-provisioned with network numbers and a "here is your default gateway" configuration, and that gateway were to lose power, the phones would lose connectivity, and might not recover until they discarded their DHCP credentials and routing information, and broadcast for a new configuration... which would happen if they were power-cycled, or (if not then) many hours later. Similar things could happen if (for example) a janitor were to plug a floor polisher into a power circuit shared with servers or network equipment... the turn-on / turn-off power sags and spikes might knock networking gear off-line. [This is not a hypothetical example... numerous cases of this sort of thing have been reported over the years.] If these phones are being DHCP-provisioned, you might want to check each phone and see what configuration has been acquired... i.e. if it got its information from the "real" DHCP server, or from some other source. I've had network problems in the past result from people plugging some sort of "all-in-one" appliance or server into an existing net... the appliance starts trying to provide DHCP service and routing on its own. This can seriously disrupt the network... either immediately (if the appliance's configuration is incompatible with the network) or at a later time (if e.g. the appliance is acting as a router, and is then powered off). -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
On Thu, Jul 21, 2011 at 7:13 PM, Mike Diehl wrote: > Hi all, > > I've got a strange problem with a customer's phones. > > They've got a bunch of Grandstreams that seem to be rock solid... until > 7:00pm. At 7:00, some of the phones become unavailable, and stay down. > Call > quality is solid almost all the time. But right at 7:00, things go bad. > Only > some of the phone lines go down and they stay down until the phone is > rebooted. > > I'm not even sure what to look for when I go to the site. Any ideas? > > Many years ago, in my college days, the network in one building would fail around a certain time every day. The sun would hit the network closet around the same time every day in the summer, causing the equipment to overheat and temporarily fail. I would go there and observe everything which happens at 7:00. Maybe it's something that a cleaning service inadvertently does, like faulty wiring + a vacuum cleaner. -M -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
On Friday 22 July 2011 1:42:33 am Ishfaq Malik wrote: > On Thu, 2011-07-21 at 17:13 -0600, Mike Diehl wrote: > > Hi all, > > > > I've got a strange problem with a customer's phones. > > > > They've got a bunch of Grandstreams that seem to be rock solid... until > > 7:00pm. At 7:00, some of the phones become unavailable, and stay down. > > Call quality is solid almost all the time. But right at 7:00, things go > > bad. Only some of the phone lines go down and they stay down until the > > phone is rebooted. > > > > I'm not even sure what to look for when I go to the site. Any ideas? > > Are the phones running on the same connection as the computers/servers? > If so, are they doing any scheduled backups? > > Ish No, the phones are in a different city than the servers and connected via the Internet. The users probably aren't sophisticated enough to be running a scheduled backup... -- Take care and have fun, Mike Diehl. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Strange network issue
On Thu, 2011-07-21 at 17:13 -0600, Mike Diehl wrote: > Hi all, > > I've got a strange problem with a customer's phones. > > They've got a bunch of Grandstreams that seem to be rock solid... until > 7:00pm. At 7:00, some of the phones become unavailable, and stay down. Call > quality is solid almost all the time. But right at 7:00, things go bad. > Only > some of the phone lines go down and they stay down until the phone is > rebooted. > > I'm not even sure what to look for when I go to the site. Any ideas? > Are the phones running on the same connection as the computers/servers? If so, are they doing any scheduled backups? Ish -- Ishfaq Malik Software Developer PackNet Ltd Office: 0161 660 3062 -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
[asterisk-users] Strange network issue
Hi all, I've got a strange problem with a customer's phones. They've got a bunch of Grandstreams that seem to be rock solid... until 7:00pm. At 7:00, some of the phones become unavailable, and stay down. Call quality is solid almost all the time. But right at 7:00, things go bad. Only some of the phone lines go down and they stay down until the phone is rebooted. I'm not even sure what to look for when I go to the site. Any ideas? -- Take care and have fun, Mike Diehl. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
