Hi, I found a bug while working on a route server implementation based on OpenBGPD. I have a IPv6 session from OpenBGPD 4.4 (on OpenBSD 4.4, routeertnix) to Quagga 0.99.5 (laborantix).
I have multiple IPv4 peers, and multiple IPv6 peers in the setup. When I start the BGP daemon, everything starts up nicely. All sessions come up. When I clear a IPv6 peering session, the connection shifts to the Idle state. When I look in the log, I can see it connect and establish a connection, but break as soon as a mistery update gets send out. While looking in to the problem, we found out that OpenBGPD sends a empty UPDATE, on which quagga responds by terminating the process. The /var/log/daemon log shows the following: Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor 2001:db8:1::a506:5502:1 (laborantix ipv6): state change Idle -> Connect, reason: Start Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor 2001:db8:1::a506:5502:1 (laborantix ipv6): state change Connect -> OpenSent, reason: Connection opened Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor 2001:db8:1::a506:5502:1 (laborantix ipv6): state change OpenSent -> OpenConfirm, reason: OPEN message received Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor 2001:db8:1::a506:5502:1 (laborantix ipv6): state change OpenConfirm -> Established, reason: KEEPALIVE message received Jan 29 16:07:40 routeertnix bgpd[16710]: neighbor 2001:db8:1::a506:5502:1 (laborantix ipv6) AS65502: update 2001:db8:97::/64 via 2001:db8:1::a506:5502:1 Jan 29 16:07:40 routeertnix bgpd[25774]: nexthop 2001:db8:1::a506:5502:1 now valid: directly connected Jan 29 16:07:40 routeertnix bgpd[31121]: neighbor 2001:db8:1::a506:5502:1 (laborantix ipv6): received notification: error in UPDATE message, network unacceptable Jan 29 16:07:40 routeertnix bgpd[31121]: neighbor 2001:db8:1::a506:5502:1 (laborantix ipv6): state change Established -> Idle, reason: NOTIFICATION received While doing a tcpdump we found the following packets leading to a NOTIFICATION. As you can see, frame 19 is an empty UPDATE packet. Frame 18 (167 bytes on wire, 167 bytes captured) Arrival Time: Jan 29, 2009 15:54:28.184019000 [Time delta from previous packet: 0.807505000 seconds] [Time since reference or first frame: 1.009967000 seconds] Frame Number: 18 Packet Length: 167 bytes Capture Length: 167 bytes [Frame is marked: False] [Protocols in frame: eth:ipv6:tcp:bgp] Ethernet II, Src: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec), Dst: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) Destination: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) Address: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Source: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec) Address: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Type: IPv6 (0x86dd) Internet Protocol Version 6 Version: 6 Traffic class: 0x00 Flowlabel: 0x00000 Payload length: 113 Next header: TCP (0x06) Hop limit: 1 Source address: 2001:db8:1::a506:5502:1 (2001:db8:1::a506:5502:1) Destination address: 2001:db8:1::a500:6777:1 (2001:db8:1::a500:6777:1) Transmission Control Protocol, Src Port: 179 (179), Dst Port: 10379 (10379), Seq: 84, Ack: 229, Len: 81 Source port: 179 (179) Destination port: 10379 (10379) Sequence number: 84 (relative sequence number) [Next sequence number: 165 (relative sequence number)] Acknowledgement number: 229 (relative ack number) Header length: 32 bytes Flags: 0x18 (PSH, ACK) 0... .... = Congestion Window Reduced (CWR): Not set .0.. .... = ECN-Echo: Not set ..0. .... = Urgent: Not set ...1 .... = Acknowledgment: Set .... 1... = Push: Set .... .0.. = Reset: Not set .... ..0. = Syn: Not set .... ...0 = Fin: Not set Window size: 5712 Checksum: 0x626e [incorrect, should be 0xc328 (maybe caused by checksum offloading?)] Options: (12 bytes) NOP NOP Timestamps: TSval 2877490800, TSecr 257498766 [SEQ/ACK analysis] [This is an ACK to the segment in frame: 17] [The RTT to ACK the segment was: 0.807505000 seconds] Border Gateway Protocol UPDATE Message Marker: 16 bytes Length: 81 bytes Type: UPDATE Message (2) Unfeasible routes length: 0 bytes Total path attribute length: 58 bytes Path attributes ORIGIN: IGP (4 bytes) Flags: 0x40 (Well-known, Transitive, Complete) 0... .... = Well-known .1.. .... = Transitive ..0. .... = Complete ...0 .... = Regular length Type code: ORIGIN (1) Length: 1 byte Origin: IGP (0) AS_PATH: 65502 (7 bytes) Flags: 0x40 (Well-known, Transitive, Complete) 0... .... = Well-known .1.. .... = Transitive ..0. .... = Complete ...0 .... = Regular length Type code: AS_PATH (2) Length: 4 bytes AS path: 65502 AS path segment: 65502 Path segment type: AS_SEQUENCE (2) Path segment length: 1 AS Path segment value: 65502 MULTI_EXIT_DISC: 0 (7 bytes) Flags: 0x80 (Optional, Non-transitive, Complete) 1... .... = Optional .0.. .... = Non-transitive ..0. .... = Complete ...0 .... = Regular length Type code: MULTI_EXIT_DISC (4) Length: 4 bytes Multiple exit discriminator: 0 COMMUNITIES: 6777:6777 (7 bytes) Flags: 0xc0 (Optional, Transitive, Complete) 1... .... = Optional .1.. .... = Transitive ..0. .... = Complete ...0 .... = Regular length Type code: COMMUNITIES (8) Length: 4 bytes Communities: 6777:6777 Community: 6777:6777 Community AS: 6777 Community value: 6777 MP_REACH_NLRI (33 bytes) Flags: 0x80 (Optional, Non-transitive, Complete) 1... .... = Optional .0.. .... = Non-transitive ..0. .... = Complete ...0 .... = Regular length Type code: MP_REACH_NLRI (14) Length: 30 bytes Address family: IPv6 (2) Subsequent address family identifier: Unicast (1) Next hop network address (16 bytes) Next hop: 2001:db8:1::a506:5502:1 (16) Subnetwork points of attachment: 0 Network layer reachability information (9 bytes) 2001:db8:97::/64 MP Reach NLRI prefix length: 64 MP Reach NLRI prefix: 2001:db8:97:: Frame 19 (124 bytes on wire, 124 bytes captured) Arrival Time: Jan 29, 2009 15:54:28.185078000 [Time delta from previous packet: 0.001059000 seconds] [Time since reference or first frame: 1.011026000 seconds] Frame Number: 19 Packet Length: 124 bytes Capture Length: 124 bytes [Frame is marked: False] [Protocols in frame: eth:ipv6:tcp:bgp] Ethernet II, Src: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2), Dst: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec) Destination: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec) Address: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Source: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) Address: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Type: IPv6 (0x86dd) Internet Protocol Version 6 Version: 6 Traffic class: 0x00 Flowlabel: 0x809f0 Payload length: 70 Next header: TCP (0x06) Hop limit: 1 Source address: 2001:db8:1::a500:6777:1 (2001:db8:1::a500:6777:1) Destination address: 2001:db8:1::a506:5502:1 (2001:db8:1::a506:5502:1) Transmission Control Protocol, Src Port: 10379 (10379), Dst Port: 179 (179), Seq: 229, Ack: 165, Len: 38 Source port: 10379 (10379) Destination port: 179 (179) Sequence number: 229 (relative sequence number) [Next sequence number: 267 (relative sequence number)] Acknowledgement number: 165 (relative ack number) Header length: 32 bytes Flags: 0x18 (PSH, ACK) 0... .... = Congestion Window Reduced (CWR): Not set .0.. .... = ECN-Echo: Not set ..0. .... = Urgent: Not set ...1 .... = Acknowledgment: Set .... 1... = Push: Set .... .0.. = Reset: Not set .... ..0. = Syn: Not set .... ...0 = Fin: Not set Window size: 16384 Checksum: 0xe291 [correct] Options: (12 bytes) NOP NOP Timestamps: TSval 257498768, TSecr 2877490800 [SEQ/ACK analysis] [This is an ACK to the segment in frame: 18] [The RTT to ACK the segment was: 0.001059000 seconds] Border Gateway Protocol UPDATE Message Marker: 16 bytes Length: 38 bytes Type: UPDATE Message (2) Unfeasible routes length: 0 bytes Total path attribute length: 0 bytes Frame 20 (107 bytes on wire, 107 bytes captured) Arrival Time: Jan 29, 2009 15:54:28.185337000 [Time delta from previous packet: 0.000259000 seconds] [Time since reference or first frame: 1.011285000 seconds] Frame Number: 20 Packet Length: 107 bytes Capture Length: 107 bytes [Frame is marked: False] [Protocols in frame: eth:ipv6:tcp:bgp] Ethernet II, Src: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec), Dst: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) Destination: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) Address: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Source: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec) Address: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Type: IPv6 (0x86dd) Internet Protocol Version 6 Version: 6 Traffic class: 0x00 Flowlabel: 0x00000 Payload length: 53 Next header: TCP (0x06) Hop limit: 1 Source address: 2001:db8:1::a506:5502:1 (2001:db8:1::a506:5502:1) Destination address: 2001:db8:1::a500:6777:1 (2001:db8:1::a500:6777:1) Transmission Control Protocol, Src Port: 179 (179), Dst Port: 10379 (10379), Seq: 165, Ack: 267, Len: 21 Source port: 179 (179) Destination port: 10379 (10379) Sequence number: 165 (relative sequence number) [Next sequence number: 186 (relative sequence number)] Acknowledgement number: 267 (relative ack number) Header length: 32 bytes Flags: 0x18 (PSH, ACK) 0... .... = Congestion Window Reduced (CWR): Not set .0.. .... = ECN-Echo: Not set ..0. .... = Urgent: Not set ...1 .... = Acknowledgment: Set .... 1... = Push: Set .... .0.. = Reset: Not set .... ..0. = Syn: Not set .... ...0 = Fin: Not set Window size: 5712 Checksum: 0x6232 [incorrect, should be 0xbdaa (maybe caused by checksum offloading?)] Options: (12 bytes) NOP NOP Timestamps: TSval 2877490800, TSecr 257498768 [SEQ/ACK analysis] [This is an ACK to the segment in frame: 19] [The RTT to ACK the segment was: 0.000259000 seconds] Border Gateway Protocol NOTIFICATION Message Marker: 16 bytes Length: 21 bytes Type: NOTIFICATION Message (3) Error code: UPDATE Message Error (3) Error subcode: Invalid Network Field (10) When the NOTIFICATION is received, the peer is set back to the state Idle, where the process starts again. The only way to break the cicle is to restart the entire OpenBGPD daemon. Kind regards, Arnoud Vermeer