The number of entries in the debug log for "ICMP pkt not handled from xx.xx.xx.xx" is a little suspicious. This debug message means that InterMapper received an ICMP echo reply when there was no device on any map expecting it. Under normal circumstances, this can happen if you close a map or remove a device while a ping is "in flight". The map is closed when the ping reply comes back, so the packet is recorded as "not handled". The "ICMP pkt not handled" message can also be logged if the ping reply is really late or there are two or more replies to a single echo request.
The number of "ICMP pkt not handled" messages does not correlate to your significant packet loss, however.
What happens to InterMapper's packet loss measurements if you triple the probe timeout from 3 seconds to 9 seconds for one of the ping devices?
Thanks,
Bill Fisher Dartware, LLC
On Jan 24, 2005, at 2:53 PM, Steven Good wrote:
Bill,
Here you go....
I dumped the data from the snoop using the "-v" option. It looks like all
the data you need is available with the snoop -v option (I was using the
-V before that is a bit less verbose).
Since I had quite a lot of snoop data, I am only including the time frame
that includes one good ICMP poll, one failed (three retires) poll, then
one more good poll that occurs after the failed poll period.
The Debug grep data follows after the snoop data.
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 21 arrived at 9:36:50.78 ETHER: Packet size = 62 bytes ETHER: Destination = 0:0:c:7:ac:1, Cisco ETHER: Source = 8:0:20:c4:e:56, Sun ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 48 bytes IP: Identification = 40892 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 1 (ICMP) IP: Header checksum = 91d5 IP: Source address = 129.21.6.1, rit010n19 IP: Destination address = 129.21.66.15, x009-066m-15n3.rit.edu IP: No options IP: ICMP: ----- ICMP Header ----- ICMP: ICMP: Type = 8 (Echo request) ICMP: Code = 0 (ID: 27552 Sequence number: 43572) ICMP: Checksum = 9cc8 ICMP:
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 22 arrived at 9:36:50.78 ETHER: Packet size = 62 bytes ETHER: Destination = 8:0:20:c4:e:56, Sun ETHER: Source = aa:0:4:0:fd:7, DEC (DECNET) ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 48 bytes IP: Identification = 40892 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 252 seconds/hops IP: Protocol = 1 (ICMP) IP: Header checksum = 94d5 IP: Source address = 129.21.66.15, x009-066m-15n3.rit.edu IP: Destination address = 129.21.6.1, rit010n19 IP: No options IP: ICMP: ----- ICMP Header ----- ICMP: ICMP: Type = 0 (Echo reply) ICMP: Code = 0 (ID: 27552 Sequence number: 43572) ICMP: Checksum = a4c8 ICMP:
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 23 arrived at 9:38:50.79 ETHER: Packet size = 62 bytes ETHER: Destination = 0:0:c:7:ac:1, Cisco ETHER: Source = 8:0:20:c4:e:56, Sun ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 48 bytes IP: Identification = 40895 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 1 (ICMP) IP: Header checksum = 91d2 IP: Source address = 129.21.6.1, rit010n19 IP: Destination address = 129.21.66.15, x009-066m-15n3.rit.edu IP: No options IP: ICMP: ----- ICMP Header ----- ICMP: ICMP: Type = 8 (Echo request) ICMP: Code = 0 (ID: 52560 Sequence number: 45139) ICMP: Checksum = f72 ICMP:
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 24 arrived at 9:38:53.79 ETHER: Packet size = 62 bytes ETHER: Destination = 0:0:c:7:ac:1, Cisco ETHER: Source = 8:0:20:c4:e:56, Sun ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 48 bytes IP: Identification = 40896 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 1 (ICMP) IP: Header checksum = 91d1 IP: Source address = 129.21.6.1, rit010n19 IP: Destination address = 129.21.66.15, x009-066m-15n3.rit.edu IP: No options IP: ICMP: ----- ICMP Header ----- ICMP: ICMP: Type = 8 (Echo request) ICMP: Code = 0 (ID: 52561 Sequence number: 45172) ICMP: Checksum = b57 ICMP:
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 25 arrived at 9:38:56.79 ETHER: Packet size = 62 bytes ETHER: Destination = 0:0:c:7:ac:1, Cisco ETHER: Source = 8:0:20:c4:e:56, Sun ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 48 bytes IP: Identification = 40897 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 1 (ICMP) IP: Header checksum = 91d0 IP: Source address = 129.21.6.1, rit010n19 IP: Destination address = 129.21.66.15, x009-066m-15n3.rit.edu IP: No options IP: ICMP: ----- ICMP Header ----- ICMP: ICMP: Type = 8 (Echo request) ICMP: Code = 0 (ID: 52562 Sequence number: 45226) ICMP: Checksum = f3b ICMP:
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 26 arrived at 9:40:50.79 ETHER: Packet size = 62 bytes ETHER: Destination = 0:0:c:7:ac:1, Cisco ETHER: Source = 8:0:20:c4:e:56, Sun ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 48 bytes IP: Identification = 40900 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 1 (ICMP) IP: Header checksum = 91cd IP: Source address = 129.21.6.1, rit010n19 IP: Destination address = 129.21.66.15, x009-066m-15n3.rit.edu IP: No options IP: ICMP: ----- ICMP Header ----- ICMP: ICMP: Type = 8 (Echo request) ICMP: Code = 0 (ID: 12128 Sequence number: 46721) ICMP: Checksum = a37f ICMP:
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 27 arrived at 9:40:50.79 ETHER: Packet size = 62 bytes ETHER: Destination = 8:0:20:c4:e:56, Sun ETHER: Source = aa:0:4:0:fd:7, DEC (DECNET) ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 48 bytes IP: Identification = 40900 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 252 seconds/hops IP: Protocol = 1 (ICMP) IP: Header checksum = 94cd IP: Source address = 129.21.66.15, x009-066m-15n3.rit.edu IP: Destination address = 129.21.6.1, rit010n19 IP: No options IP: ICMP: ----- ICMP Header ----- ICMP: ICMP: Type = 0 (Echo reply) ICMP: Code = 0 (ID: 12128 Sequence number: 46721) ICMP: Checksum = ab7f ICMP:
$ egrep ICMP Debug*
Debug200501130000.txt:18:10:53 ICMP pkt not handled from 129.21.51.38, ID
= 72
Debug200501130000.txt:21:36:01 ICMP pkt not handled from 129.21.51.62, ID
= 6fa2
Debug200501130000.txt:21:57:40 ICMP pkt not handled from 129.21.51.51, ID
= bba2
Debug200501130000.txt:23:55:12 ICMP pkt not handled from 129.21.1.18, ID =
ad02
Debug200501130000.txt:23:55:12 ICMP pkt not handled from 129.21.1.18, ID =
ad12
Debug200501140000.txt:05:32:46 ICMP pkt not handled from 129.21.51.1, ID =
c172
Debug200501140000.txt:08:56:09 ICMP pkt not handled from 129.21.51.65, ID
= 80b2
Debug200501140000.txt:09:46:36 ICMP pkt not handled from 129.21.51.73, ID
= 82f2
Debug200501140000.txt:13:14:49 ICMP pkt not handled from 129.21.51.9, ID =
e6d2
Debug200501140000.txt:16:15:28 ICMP pkt not handled from 129.21.51.9, ID =
57c2
Debug200501140000.txt:17:30:12 ICMP pkt not handled from 129.21.1.18, ID =
5792
Debug200501140000.txt:17:46:41 ICMP pkt not handled from 129.21.51.8, ID =
2572
Debug200501140000.txt:22:55:24 ICMP pkt not handled from 129.21.51.83, ID
= e52
Debug200501150000.txt:01:35:12 ICMP pkt not handled from 129.21.1.18, ID =
bda2
Debug200501150000.txt:01:35:12 ICMP pkt not handled from 129.21.1.18, ID =
bdb2
Debug200501150000.txt:02:39:34 ICMP pkt not handled from 129.21.51.82, ID
= bbe2
Debug200501150000.txt:07:40:10 ICMP pkt not handled from 129.21.1.18, ID =
61a2
Debug200501150000.txt:07:40:10 ICMP pkt not handled from 129.21.1.18, ID =
61b2
Debug200501150000.txt:09:21:09 ICMP pkt not handled from 129.21.51.89, ID
= b6f2
Debug200501150000.txt:10:48:53 ICMP pkt not handled from 129.21.51.85, ID
= 4802
Debug200501150000.txt:14:43:44 ICMP pkt not handled from 129.21.51.86, ID
= 9292
Debug200501150000.txt:18:05:11 ICMP pkt not handled from 129.21.1.18, ID =
c8b2
Debug200501150000.txt:18:05:11 ICMP pkt not handled from 129.21.1.18, ID =
c8d2
Debug200501160000.txt:05:03:08 ICMP pkt not handled from 129.21.51.16, ID
= c072
Debug200501160000.txt:12:47:37 ICMP pkt not handled from 129.21.51.18, ID
= 4992
Debug200501160000.txt:17:02:29 ICMP pkt not handled from 129.21.51.5, ID =
e152
Debug200501160000.txt:21:08:08 ICMP pkt not handled from 129.21.51.42, ID
= a710
Debug200501160000.txt:22:23:57 ICMP pkt not handled from 129.21.51.24, ID
= c3f2
Debug200501170000.txt:01:48:55 ICMP pkt not handled from 129.21.51.4, ID =
e762
Debug200501170000.txt:02:24:43 ICMP pkt not handled from 129.21.51.24, ID
= 5482
Debug200501170000.txt:04:48:21 ICMP pkt not handled from 129.21.51.3, ID =
5d2
Debug200501170000.txt:05:13:15 ICMP pkt not handled from 129.21.51.84, ID
= 8792
Debug200501170000.txt:13:57:37 ICMP pkt not handled from 129.21.51.81, ID
= 3692
Debug200501170000.txt:16:06:21 ICMP pkt not handled from 129.21.51.30, ID
= 3372
Debug200501170000.txt:19:58:52 ICMP pkt not handled from 129.21.51.85, ID
= ad2
Debug200501170000.txt:22:51:14 ICMP pkt not handled from 129.21.51.3, ID =
e032
Debug200501180000.txt:02:42:25 ICMP pkt not handled from 129.21.155.253,
ID = dd22
Debug200501180000.txt:02:53:57 ICMP pkt not handled from 129.21.169.252,
ID = 33d2
Debug200501180000.txt:04:23:31 ICMP pkt not handled from 129.21.51.36, ID
= a530
Debug200501180000.txt:06:34:25 ICMP pkt not handled from 129.21.51.89, ID
= a622
Debug200501180000.txt:08:09:58 ICMP pkt not handled from 129.21.51.80, ID
= 6a32
Debug200501180000.txt:12:56:06 ICMP pkt not handled from 129.21.51.86, ID
= a1a2
Debug200501180000.txt:17:16:52 ICMP pkt not handled from 129.21.51.65, ID
= 8332
Debug200501180000.txt:20:10:15 ICMP pkt not handled from 129.21.1.18, ID =
3ed2
Debug200501180000.txt:20:10:15 ICMP pkt not handled from 129.21.1.18, ID =
3ee2
Debug200501180000.txt:20:13:59 ICMP pkt not handled from 129.21.51.83, ID
= dd92
Debug200501180000.txt:20:35:13 ICMP pkt not handled from 129.21.1.18, ID =
34d2
Debug200501180000.txt:20:35:13 ICMP pkt not handled from 129.21.1.18, ID =
34e2
Debug200501190000.txt:06:35:12 ICMP pkt not handled from 129.21.1.18, ID =
5082
Debug200501190000.txt:06:35:12 ICMP pkt not handled from 129.21.1.18, ID =
5092
Debug200501190000.txt:09:36:32 ICMP pkt not handled from 129.21.51.24, ID
= 76a2
Debug200501190000.txt:10:59:30 ICMP pkt not handled from 129.21.51.82, ID
= 78d2
Debug200501190000.txt:11:59:47 ICMP pkt not handled from 129.21.51.82, ID
= 5c52
Debug200501190000.txt:13:55:34 ICMP pkt not handled from 129.21.51.1, ID =
cad2
Debug200501190000.txt:17:09:04 ICMP pkt not handled from 129.21.51.10, ID
= d542
Debug200501190000.txt:17:25:12 ICMP pkt not handled from 129.21.1.18, ID =
1a12
Debug200501190000.txt:17:25:12 ICMP pkt not handled from 129.21.1.18, ID =
1a22
Debug200501190000.txt:19:05:11 ICMP pkt not handled from 129.21.1.18, ID =
6f72
Debug200501190000.txt:19:05:11 ICMP pkt not handled from 129.21.1.18, ID =
6f82
Debug200501190000.txt:19:08:48 ICMP pkt not handled from 129.21.51.11, ID
= 1142
Debug200501190000.txt:19:15:13 ICMP pkt not handled from 129.21.1.18, ID =
e22
Debug200501190000.txt:19:15:13 ICMP pkt not handled from 129.21.1.18, ID =
e32
Debug200501190000.txt:20:10:13 ICMP pkt not handled from 129.21.1.18, ID =
f572
Debug200501190000.txt:20:10:13 ICMP pkt not handled from 129.21.1.18, ID =
f582
Debug200501190000.txt:21:30:11 ICMP pkt not handled from 129.21.1.18, ID =
8ff2
Debug200501190000.txt:21:30:11 ICMP pkt not handled from 129.21.1.18, ID =
9002
Debug200501190000.txt:21:39:08 ICMP pkt not handled from 129.21.51.49, ID
= fc22
Debug200501190000.txt:23:09:38 ICMP pkt not handled from 129.21.51.11, ID
= 5d02
Debug200501190000.txt:23:25:12 ICMP pkt not handled from 129.21.1.18, ID =
50b2
Debug200501190000.txt:23:25:12 ICMP pkt not handled from 129.21.1.18, ID =
50c2
Debug200501200000.txt:01:00:11 ICMP pkt not handled from 129.21.1.18, ID =
ef2
Debug200501200000.txt:02:30:12 ICMP pkt not handled from 129.21.1.18, ID =
a792
Debug200501200000.txt:02:30:12 ICMP pkt not handled from 129.21.1.18, ID =
a7b2
Debug200501200000.txt:05:24:12 ICMP pkt not handled from 129.21.51.65, ID
= 7022
Debug200501200000.txt:15:17:18 ICMP pkt not handled from 129.21.1.30, ID =
5600
Debug200501200000.txt:16:50:53 ICMP pkt not handled from 129.21.51.53, ID
= bcc2
Debug200501200000.txt:17:40:12 ICMP pkt not handled from 129.21.1.18, ID =
1922
Debug200501200000.txt:19:48:05 ICMP pkt not handled from 129.21.51.8, ID =
6802
Debug200501200000.txt:XSocketICMP_BSD.cp: sendto errno 11
Debug200501210000.txt:06:07:29 ICMP pkt not handled from 129.21.51.82, ID
= 48d2
Debug200501210000.txt:17:23:10 ICMP pkt not handled from 129.21.51.101, ID
= 18b0
Debug200501220000.txt:02:46:27 ICMP pkt not handled from 129.21.51.97, ID
= 810
Debug200501230000.txt:23:56:55 ICMP pkt not handled from 129.21.51.92, ID
= 17c0
Debug200501240000.txt:00:57:10 ICMP pkt not handled from 129.21.51.108, ID
= 8c90
Debug200501240000.txt:13:38:09 ICMP pkt not handled from 129.21.51.111, ID
= 9ed0
Thanks,
Steve
-----Original Message-----
From: [email protected]
[mailto:[EMAIL PROTECTED] On Behalf Of William W. Fisher
Sent: Monday, January 24, 2005 2:07 PM
To: InterMapper Discussion
Subject: Re: [IM-Talk] DOWN outages because of no response to ICMP Echo
polling - has anyone else had this type of problem
On Jan 24, 2005, at 12:39 PM, Steven Good wrote:
I am including the snoop data. I already removed any lines not relevant to this device. I am only including the snoop for this time period showing a successful poll both before and after the poll with out replies.
Steven:
Can you collect a more detailed trace showing the ICMP requests and responses, perhaps using tcpdump to a file? There are some things I would like to check in the outgoing ICMP echo requests:
- IP TOS - IP header checksum - ICMP checksum etc.
These are things in the echo request where a misstep would lead to no response. I would also recommend grep'ing InterMapper's debug log for "ICMP" to check if any API errors have been logged.
Thanks,
Bill Fisher Dartware, LLC
____________________________________________________________________ List archives: http://www.mail-archive.com/intermapper-talk%40list.dartware.com/ To unsubscribe: send email to: [EMAIL PROTECTED]
____________________________________________________________________
List archives: http://www.mail-archive.com/intermapper-talk%40list.dartware.com/
To unsubscribe: send email to: [EMAIL PROTECTED]
