Re: bgpd fails to install ipv6 routes in kernel routing table
On Mon, Feb 09, 2009 at 04:51:12PM +1100, Graeme Lee wrote:
Graeme Lee wrote:
Graeme Lee wrote:
tico wrote:
Graeme Lee wrote:
tico wrote:
Graeme Lee wrote:
snip
Ok forget bgp configs for a minute. I've been quickly scanning over
the code, and notable is that the log displays:
Feb 9 13:00:15 gw-nextgen bgpd[17223]: send_rtmsg: action 1, prefix
2001:7fb:fe07::/48: Network is unreachable
but shouldn't it be a send_rt6msg call in kroute.c?
Yes. The waning message had the wrong function name in it.
On a hunch, I tried a 64bit and a 32 bit machine with 1 prefix each.
The 32bit machine adds routes to the kernel without complaint. The
64bit machine complained with send_rtmsg
Arrg. IPv6 is once again broken by design. For some ridiculous reason
struct sockaddr_in6's size is 28 bytes. So IPv6 fucks up alignment on 64 bit
archs. All hail link local addressing and all the crappy workarounds
needed for it.
Please try the attached diff.
--
:wq Claudio
Index: kroute.c
===
RCS file: /cvs/src/usr.sbin/bgpd/kroute.c,v
retrieving revision 1.164
diff -u -p -r1.164 kroute.c
--- kroute.c9 Feb 2009 08:20:11 - 1.164
+++ kroute.c9 Feb 2009 08:49:00 -
@@ -2057,12 +2057,14 @@ retry:
int
send_rt6msg(int fd, int action, struct kroute6 *kroute)
{
- struct ioveciov[5];
+ struct ioveciov[8];
struct rt_msghdrhdr;
struct sockaddr_in6 prefix;
struct sockaddr_in6 nexthop;
struct sockaddr_in6 mask;
struct sockaddr_rtlabel label;
+ chargrmbl[sizeof(long) - (sizeof(prefix)
+ (sizeof(long) - 1))];
int iovcnt = 0;
if (kr_state.fib_sync == 0)
@@ -2070,6 +2072,7 @@ send_rt6msg(int fd, int action, struct k
/* initialize header */
bzero(hdr, sizeof(hdr));
+ bzero(grmbl, sizeof(grmbl));
hdr.rtm_version = RTM_VERSION;
hdr.rtm_type = action;
hdr.rtm_tableid = kr_state.rtableid;
@@ -2096,6 +2099,11 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = prefix;
iov[iovcnt++].iov_len = sizeof(prefix);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
+
if (memcmp(kroute-nexthop, in6addr_any, sizeof(struct in6_addr))) {
bzero(nexthop, sizeof(nexthop));
@@ -2110,6 +2118,10 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = nexthop;
iov[iovcnt++].iov_len = sizeof(nexthop);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
}
bzero(mask, sizeof(mask));
@@ -2123,6 +2135,10 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = mask;
iov[iovcnt++].iov_len = sizeof(mask);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
if (kroute-labelid) {
bzero(label, sizeof(label));
Re: bgpd fails to install ipv6 routes in kernel routing table
Claudio Jeker wrote: On Mon, Feb 09, 2009 at 04:51:12PM +1100, Graeme Lee wrote: Graeme Lee wrote: Graeme Lee wrote: tico wrote: Graeme Lee wrote: tico wrote: Graeme Lee wrote: snip Ok forget bgp configs for a minute. I've been quickly scanning over the code, and notable is that the log displays: Feb 9 13:00:15 gw-nextgen bgpd[17223]: send_rtmsg: action 1, prefix 2001:7fb:fe07::/48: Network is unreachable but shouldn't it be a send_rt6msg call in kroute.c? Yes. The waning message had the wrong function name in it. well I was looking at least. On a hunch, I tried a 64bit and a 32 bit machine with 1 prefix each. The 32bit machine adds routes to the kernel without complaint. The 64bit machine complained with send_rtmsg Arrg. IPv6 is once again broken by design. For some ridiculous reason struct sockaddr_in6's size is 28 bytes. So IPv6 fucks up alignment on 64 bit archs. All hail link local addressing and all the crappy workarounds needed for it. Please try the attached diff. You are altogether a legend. I now have the full ipv6 table in the kernel.
Re: bgpd fails to install ipv6 routes in kernel routing table
Am 09.02.2009 um 09:53 schrieb Claudio Jeker: Please try the attached diff. A general question about diffs like this: will these diffs automatically go to -current in the next couple of days/weeks? Or do I have to apply all these patches by hand? :wq Claudio Thanks, Falk
Re: bgpd fails to install ipv6 routes in kernel routing table
On Mon, Feb 9, 2009 at 12:53 AM, Claudio Jeker cje...@diehard.n-r-g.com wrote:
On a hunch, I tried a 64bit and a 32 bit machine with 1 prefix each.
The 32bit machine adds routes to the kernel without complaint. The
64bit machine complained with send_rtmsg
Arrg. IPv6 is once again broken by design. For some ridiculous reason
struct sockaddr_in6's size is 28 bytes. So IPv6 fucks up alignment on 64 bit
archs. All hail link local addressing and all the crappy workarounds
needed for it.
Maybe it is too late for me to be thinking about this ... but could
you explain the diff below? Unless I'm missing something obvious, it
looks like it changes behavior for non-64bit archs as well.
--patrick
Please try the attached diff.
--
:wq Claudio
Index: kroute.c
===
RCS file: /cvs/src/usr.sbin/bgpd/kroute.c,v
retrieving revision 1.164
diff -u -p -r1.164 kroute.c
--- kroute.c9 Feb 2009 08:20:11 - 1.164
+++ kroute.c9 Feb 2009 08:49:00 -
@@ -2057,12 +2057,14 @@ retry:
int
send_rt6msg(int fd, int action, struct kroute6 *kroute)
{
- struct ioveciov[5];
+ struct ioveciov[8];
struct rt_msghdrhdr;
struct sockaddr_in6 prefix;
struct sockaddr_in6 nexthop;
struct sockaddr_in6 mask;
struct sockaddr_rtlabel label;
+ chargrmbl[sizeof(long) - (sizeof(prefix)
+ (sizeof(long) - 1))];
int iovcnt = 0;
if (kr_state.fib_sync == 0)
@@ -2070,6 +2072,7 @@ send_rt6msg(int fd, int action, struct k
/* initialize header */
bzero(hdr, sizeof(hdr));
+ bzero(grmbl, sizeof(grmbl));
hdr.rtm_version = RTM_VERSION;
hdr.rtm_type = action;
hdr.rtm_tableid = kr_state.rtableid;
@@ -2096,6 +2099,11 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = prefix;
iov[iovcnt++].iov_len = sizeof(prefix);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
+
if (memcmp(kroute-nexthop, in6addr_any, sizeof(struct in6_addr))) {
bzero(nexthop, sizeof(nexthop));
@@ -2110,6 +2118,10 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = nexthop;
iov[iovcnt++].iov_len = sizeof(nexthop);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
}
bzero(mask, sizeof(mask));
@@ -2123,6 +2135,10 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = mask;
iov[iovcnt++].iov_len = sizeof(mask);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
if (kroute-labelid) {
bzero(label, sizeof(label));
Re: bgpd fails to install ipv6 routes in kernel routing table
On Mon, Feb 09, 2009 at 10:13:42AM +0100, Falk Brockerhoff - smartTERRA GmbH wrote: Am 09.02.2009 um 09:53 schrieb Claudio Jeker: Please try the attached diff. A general question about diffs like this: will these diffs automatically go to -current in the next couple of days/weeks? Or do I have to apply all these patches by hand? If the diff works it will go into -current. So currently I'm waiting for positive test results and hopefully an ok by henning@ -- :wq Claudio
Re: bgpd fails to install ipv6 routes in kernel routing table
On Mon, Feb 09, 2009 at 02:22:08AM -0800, patrick keshishian wrote:
On Mon, Feb 9, 2009 at 12:53 AM, Claudio Jeker cje...@diehard.n-r-g.com
wrote:
On a hunch, I tried a 64bit and a 32 bit machine with 1 prefix each.
The 32bit machine adds routes to the kernel without complaint. The
64bit machine complained with send_rtmsg
Arrg. IPv6 is once again broken by design. For some ridiculous reason
struct sockaddr_in6's size is 28 bytes. So IPv6 fucks up alignment on 64 bit
archs. All hail link local addressing and all the crappy workarounds
needed for it.
Maybe it is too late for me to be thinking about this ... but could
you explain the diff below? Unless I'm missing something obvious, it
looks like it changes behavior for non-64bit archs as well.
Hmm. I think your right. I think a different approach would be better.
Will cook up something later today.
--patrick
Please try the attached diff.
--
:wq Claudio
Index: kroute.c
===
RCS file: /cvs/src/usr.sbin/bgpd/kroute.c,v
retrieving revision 1.164
diff -u -p -r1.164 kroute.c
--- kroute.c9 Feb 2009 08:20:11 - 1.164
+++ kroute.c9 Feb 2009 08:49:00 -
@@ -2057,12 +2057,14 @@ retry:
int
send_rt6msg(int fd, int action, struct kroute6 *kroute)
{
- struct ioveciov[5];
+ struct ioveciov[8];
struct rt_msghdrhdr;
struct sockaddr_in6 prefix;
struct sockaddr_in6 nexthop;
struct sockaddr_in6 mask;
struct sockaddr_rtlabel label;
+ chargrmbl[sizeof(long) - (sizeof(prefix)
+ (sizeof(long) - 1))];
int iovcnt = 0;
if (kr_state.fib_sync == 0)
@@ -2070,6 +2072,7 @@ send_rt6msg(int fd, int action, struct k
/* initialize header */
bzero(hdr, sizeof(hdr));
+ bzero(grmbl, sizeof(grmbl));
hdr.rtm_version = RTM_VERSION;
hdr.rtm_type = action;
hdr.rtm_tableid = kr_state.rtableid;
@@ -2096,6 +2099,11 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = prefix;
iov[iovcnt++].iov_len = sizeof(prefix);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
+
if (memcmp(kroute-nexthop, in6addr_any, sizeof(struct in6_addr)))
{
bzero(nexthop, sizeof(nexthop));
@@ -2110,6 +2118,10 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = nexthop;
iov[iovcnt++].iov_len = sizeof(nexthop);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
}
bzero(mask, sizeof(mask));
@@ -2123,6 +2135,10 @@ send_rt6msg(int fd, int action, struct k
/* adjust iovec */
iov[iovcnt].iov_base = mask;
iov[iovcnt++].iov_len = sizeof(mask);
+ /* don't we all love IPv6 */
+ hdr.rtm_msglen += sizeof(grmbl);
+ iov[iovcnt].iov_base = grmbl;
+ iov[iovcnt++].iov_len = sizeof(grmbl);
if (kroute-labelid) {
bzero(label, sizeof(label));
--
:wq Claudio
Re: bgpd fails to install ipv6 routes in kernel routing table
Am 09.02.2009 um 11:23 schrieb Claudio Jeker: If the diff works it will go into -current. So currently I'm waiting for positive test results and hopefully an ok by henning@ Perfect. Thank you (and Henning and all the others), once again, for your incredible and fast support! :wq Claudio Regards, Falk
Re: bgpd fails to install ipv6 routes in kernel routing table
On Mon, Feb 09, 2009 at 11:43:10AM +0100, Claudio Jeker wrote:
On Mon, Feb 09, 2009 at 02:22:08AM -0800, patrick keshishian wrote:
On Mon, Feb 9, 2009 at 12:53 AM, Claudio Jeker cje...@diehard.n-r-g.com
wrote:
On a hunch, I tried a 64bit and a 32 bit machine with 1 prefix each.
The 32bit machine adds routes to the kernel without complaint. The
64bit machine complained with send_rtmsg
Arrg. IPv6 is once again broken by design. For some ridiculous reason
struct sockaddr_in6's size is 28 bytes. So IPv6 fucks up alignment on 64
bit
archs. All hail link local addressing and all the crappy workarounds
needed for it.
Maybe it is too late for me to be thinking about this ... but could
you explain the diff below? Unless I'm missing something obvious, it
looks like it changes behavior for non-64bit archs as well.
Hmm. I think your right. I think a different approach would be better.
Will cook up something later today.
I think this is better. Just compile tested and no real time to test until
later today.
--
:wq Claudio
Index: kroute.c
===
RCS file: /cvs/src/usr.sbin/bgpd/kroute.c,v
retrieving revision 1.164
diff -u -p -r1.164 kroute.c
--- kroute.c9 Feb 2009 08:20:11 - 1.164
+++ kroute.c9 Feb 2009 12:36:18 -
@@ -1813,8 +1813,8 @@ inet6applymask(struct in6_addr *dest, co
dest-s6_addr[i] = src-s6_addr[i] mask.s6_addr[i];
}
-#defineROUNDUP(a, size)\
-(((a) ((size) - 1)) ? (1 + ((a) | ((size) - 1))) : (a))
+#defineROUNDUP(a) \
+(((a) ((sizeof(long)) - 1)) ? (1 + ((a) | ((sizeof(long)) - 1))) : (a))
void
get_rtaddrs(int addrs, struct sockaddr *sa, struct sockaddr **rti_info)
@@ -1825,7 +1825,7 @@ get_rtaddrs(int addrs, struct sockaddr *
if (addrs (1 i)) {
rti_info[i] = sa;
sa = (struct sockaddr *)((char *)(sa) +
- ROUNDUP(sa-sa_len, sizeof(long)));
+ ROUNDUP(sa-sa_len));
} else
rti_info[i] = NULL;
}
@@ -2059,9 +2059,10 @@ send_rt6msg(int fd, int action, struct k
{
struct ioveciov[5];
struct rt_msghdrhdr;
- struct sockaddr_in6 prefix;
- struct sockaddr_in6 nexthop;
- struct sockaddr_in6 mask;
+ struct pad {
+ struct sockaddr_in6 addr;
+ charpad[sizeof(long)];
+ } prefix, nexthop, mask;
struct sockaddr_rtlabel label;
int iovcnt = 0;
@@ -2086,43 +2087,44 @@ send_rt6msg(int fd, int action, struct k
iov[iovcnt++].iov_len = sizeof(hdr);
bzero(prefix, sizeof(prefix));
- prefix.sin6_len = sizeof(prefix);
- prefix.sin6_family = AF_INET6;
- memcpy(prefix.sin6_addr, kroute-prefix, sizeof(struct in6_addr));
+ prefix.addr.sin6_len = sizeof(struct sockaddr_in6);
+ prefix.addr.sin6_family = AF_INET6;
+ memcpy(prefix.addr.sin6_addr, kroute-prefix,
+ sizeof(struct in6_addr));
/* XXX scope does not matter or? */
/* adjust header */
hdr.rtm_addrs |= RTA_DST;
- hdr.rtm_msglen += sizeof(prefix);
+ hdr.rtm_msglen += ROUNDUP(sizeof(struct sockaddr_in6));
/* adjust iovec */
iov[iovcnt].iov_base = prefix;
- iov[iovcnt++].iov_len = sizeof(prefix);
+ iov[iovcnt++].iov_len = ROUNDUP(sizeof(struct sockaddr_in6));
if (memcmp(kroute-nexthop, in6addr_any, sizeof(struct in6_addr))) {
bzero(nexthop, sizeof(nexthop));
- nexthop.sin6_len = sizeof(nexthop);
- nexthop.sin6_family = AF_INET6;
- memcpy(nexthop.sin6_addr, kroute-nexthop,
+ nexthop.addr.sin6_len = sizeof(struct sockaddr_in6);
+ nexthop.addr.sin6_family = AF_INET6;
+ memcpy(nexthop.addr.sin6_addr, kroute-nexthop,
sizeof(struct in6_addr));
/* adjust header */
hdr.rtm_flags |= RTF_GATEWAY;
hdr.rtm_addrs |= RTA_GATEWAY;
- hdr.rtm_msglen += sizeof(nexthop);
+ hdr.rtm_msglen += ROUNDUP(sizeof(struct sockaddr_in6));
/* adjust iovec */
iov[iovcnt].iov_base = nexthop;
- iov[iovcnt++].iov_len = sizeof(nexthop);
+ iov[iovcnt++].iov_len = ROUNDUP(sizeof(struct sockaddr_in6));
}
bzero(mask, sizeof(mask));
- mask.sin6_len = sizeof(mask);
- mask.sin6_family = AF_INET6;
- memcpy(mask.sin6_addr, prefixlen2mask6(kroute-prefixlen),
+ mask.addr.sin6_len = sizeof(struct sockaddr_in6);
+ mask.addr.sin6_family = AF_INET6;
+ memcpy(mask.addr.sin6_addr, prefixlen2mask6(kroute-prefixlen),
sizeof(struct in6_addr));
/* adjust
Re: bgpd fails to install ipv6 routes in kernel routing table
Claudio Jeker wrote: On Mon, Feb 09, 2009 at 11:43:10AM +0100, Claudio Jeker wrote: On Mon, Feb 09, 2009 at 02:22:08AM -0800, patrick keshishian wrote: On Mon, Feb 9, 2009 at 12:53 AM, Claudio Jeker cje...@diehard.n-r-g.com wrote: On a hunch, I tried a 64bit and a 32 bit machine with 1 prefix each. The 32bit machine adds routes to the kernel without complaint. The 64bit machine complained with send_rtmsg Arrg. IPv6 is once again broken by design. For some ridiculous reason struct sockaddr_in6's size is 28 bytes. So IPv6 fucks up alignment on 64 bit archs. All hail link local addressing and all the crappy workarounds needed for it. Maybe it is too late for me to be thinking about this ... but could you explain the diff below? Unless I'm missing something obvious, it looks like it changes behavior for non-64bit archs as well. Hmm. I think your right. I think a different approach would be better. Will cook up something later today. I think this is better. Just compile tested and no real time to test until later today. Hi Claudio Tested on i386 and amd64 test bgp sessions ok Tested on amd64 production w/2 x ipv4 feeds and 1 x ipv6. Full ipv6 table is installed in the kernel. daemon log shows Feb 10 09:06:14 gw-nextgen bgpd[8598]: neighbor 2001:470:17:7f::1 (HurricaneHK): state change Connect - OpenSent, reason: Connection opened Feb 10 09:06:14 gw-nextgen bgpd[8598]: neighbor 2001:470:17:7f::1 (HurricaneHK): state change OpenSent - OpenConfirm, reason: OPEN message received Feb 10 09:06:14 gw-nextgen bgpd[8598]: neighbor 2001:470:17:7f::1 (HurricaneHK): state change OpenConfirm - Established, reason: KEEPALIVE message received Feb 10 09:06:18 gw-nextgen bgpd[15752]: nexthop 2001:470:17:7f::1 now valid: directly connected No errors.
Re: bgpd fails to install ipv6 routes in kernel routing table
Rogier Krieger wrote: On Sun, Feb 8, 2009 at 02:09, Graeme Lee gra...@omni.net.au wrote: The bgpd log shows this: bgpd: send_rtmsg: action 1, prefix 2001:dc8:c000::/36: Network is unreachable bgpd: send_rtmsg: action 1, prefix 2a01:a8::/32: Network is unreachable for every network received via my peer. Are there intermediate hops that you receive from the peer but cannot reach? If your nexthop is unreachable, that may explain the message. If you go back far enough in the logs (before the first prefixes you receive, the log may provide more insight as well as I don't know how many peers you have/prefixes you get). Nope. Here's the first few lines from bgpctl show ip bgp inet6 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin *2001::/32 2001:470:17:7f::1100 0 6939 12859 i *2001:200::/32 2001:470:17:7f::1100 0 6939 2500 i *2001:200:136::/48 2001:470:17:7f::1100 0 6939 2516 7660 9367 i *2001:200:600::/40 2001:470:17:7f::1100 0 6939 2516 7667 i *2001:200:900::/40 2001:470:17:7f::1100 0 6939 2516 7660 i *2001:200:a000::/35 2001:470:17:7f::1100 0 6939 3257 2497 4690 i *2001:200:c000::/35 2001:470:17:7f::1100 0 6939 2500 23634 i *2001:200:e000::/35 2001:470:17:7f::1100 0 6939 4635 7660 i *2001:208::/32 2001:470:17:7f::1100 0 6939 23911 9800 38035 7610 i *2001:218::/32 2001:470:17:7f::1100 0 6939 2914 i *2001:220::/35 2001:470:17:7f::1100 0 6939 2516 7660 9270 i *2001:220:2000::/35 2001:470:17:7f::1100 0 6939 2516 7660 9270 38128 i *2001:220:8000::/33 2001:470:17:7f::1100 0 6939 2516 7660 9270 38128 i 2001:470:17:7f::1 is my bgp peer from hurricane. The bgp table looks fine. It just doesn't translate to the kernel routing table. ergo, I cannot see or be seen. my prefix is advertised fine (2400:6800::/32) I can talk to and directly ping6 2001:470:17:7f::1 Adding static routes works (eg a default). It's just that bgpd isn't translating what it knows into the kernel. A clue to what I'm missing would be really appreciated. Other than checking the nexthop above, it'll help to include your network layout (what interfaces, uplink, addresses), bgpd configuration and a non-chopped dmesg. Dmesg was there to demonstrate I really was running -current and not something from somewhere random. Network layout is somewhat complicated. 1 x ebgp and 1 x ibgp session receive ipv4 world tables. Gif tunnel to a hurricane router in Hong Kong. I'm receiving ipv6 world bgp tables from this peer. Connectivity to the peer is fine. Just can't get past it. I can see that my prefix is announced via looking glasses. I'm receiving about 1.6k prefixes from hurricane. # bgpctl show ip bgp sum Neighbor ASMsgRcvdMsgSent OutQ Up/Down State/PrfRcvd HurricaneHK 6939 3220 1428 0 11:52:11 1588 Optus Peer 10105 104321 43663 0 11:58:08 222487 NextGen 38809 78041 1439 0 11:58:08 274913 complete restart of bgpd shows this: Feb 8 23:43:47 gw-nexgen bgpd[23344]: neighbor 2001:470:17:7f::1 (HurricaneHK): state change Connect - OpenSent, reason: Connection opened Feb 8 23:43:47 gw-nexgen bgpd[23344]: neighbor 2001:470:17:7f::1 (HurricaneHK): state change OpenSent - OpenConfirm, reason: OPEN message received Feb 8 23:43:47 gw-nexgen bgpd[23344]: neighbor 2001:470:17:7f::1 (HurricaneHK): state change OpenConfirm - Established, reason: KEEPALIVE message received Feb 8 23:44:13 gw-nexgen bgpd[4481]: nexthop 2001:470:17:7f::1 now valid: directly connected Feb 8 23:44:13 gw-nexgen bgpd[4481]: send_rtmsg: action 1, prefix 2a01:7b0::/32: Network is unreachable Feb 8 23:44:13 gw-nexgen bgpd[4481]: send_rtmsg: action 1, prefix 2404:1b0::/32: Network is unreachable Feb 8 23:44:13 gw-nexgen bgpd[4481]: send_rtmsg: action 1, prefix 2400:3000::/32: Network is unreachable etc etc for all 1.6k prefixes Hope it helps, Rogier
Re: bgpd fails to install ipv6 routes in kernel routing table
Graeme Lee wrote: snip Network layout is somewhat complicated. 1 x ebgp and 1 x ibgp session receive ipv4 world tables. Gif tunnel to a hurricane router in Hong Kong. I'm receiving ipv6 world bgp tables from this peer. Connectivity to the peer is fine. Just can't get past it. I can see that my prefix is announced via looking glasses. I'm receiving about 1.6k prefixes from hurricane. I'm speaking BGP over v6 with HE.net as well (albeit in Fremont, not HK), and I can see you just fine, and apparently you can see me (AS30708) as well, since I can ping you from both my Hurricane /64 as well as from an IP within my own /32. $ ping6 -c1 -S 2607:f618:1::1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2607:f618:1::1 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=442.275 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 442.275/442.275/442.275/0.000 ms $ ping6 -c1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2001:470:1:53::2 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=441.775 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 441.775/441.775/441.775/0.000 ms $ bgpctl sho ip bgp 2400:6800::/32 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin *2400:6800::/32 2001:470:1:53::1100 0 6939 10105 i $ uname -mr 4.4 i386 What does your bgpctl sho nex give you? -tico
Re: bgpd fails to install ipv6 routes in kernel routing table
tico wrote: Graeme Lee wrote: snip Network layout is somewhat complicated. 1 x ebgp and 1 x ibgp session receive ipv4 world tables. Gif tunnel to a hurricane router in Hong Kong. I'm receiving ipv6 world bgp tables from this peer. Connectivity to the peer is fine. Just can't get past it. I can see that my prefix is announced via looking glasses. I'm receiving about 1.6k prefixes from hurricane. I'm speaking BGP over v6 with HE.net as well (albeit in Fremont, not HK), and I can see you just fine, and apparently you can see me (AS30708) as well, since I can ping you from both my Hurricane /64 as well as from an IP within my own /32. $ ping6 -c1 -S 2607:f618:1::1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2607:f618:1::1 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=442.275 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 442.275/442.275/442.275/0.000 ms $ ping6 -c1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2001:470:1:53::2 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=441.775 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 441.775/441.775/441.775/0.000 ms $ bgpctl sho ip bgp 2400:6800::/32 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin *2400:6800::/32 2001:470:1:53::1100 0 6939 10105 i $ uname -mr 4.4 i386 What does your bgpctl sho nex give you? -tico Hi Tico. # bgpctl show next Nexthop State 2001:470:17:7f::1valid gif0UP 203.143.64.133 valid em1 UP, Ethernet, active, 100 MBit/s 121.200.227.93 valid em0 UP, Ethernet, active, 100 MBit/s However, the only reason you can see me is because i've manually stuck in a default route just to get things working # netstat -rnf inet6 Routing tables Internet6: DestinationGateway Flags Refs Use Mtu Prio Iface ::/104 ::1 UGRS 00 - 8 lo0 ::/96 ::1 UGRS 00 - 8 lo0 default2001:470:17:7f::1 UGS0 19 - 8 gif0 ::1::1 UH140 33160 4 lo0 ::127.0.0.0/104::1 UGRS 00 - 8 lo0 ::224.0.0.0/100::1 UGRS 00 - 8 lo0 ::255.0.0.0/104::1 UGRS 00 - 8 lo0 :::0.0.0.0/96 ::1 UGRS 00 - 8 lo0 2001:470:17:7f::/64link#6 UC 10 - 4 gif0 2001:470:17:7f::1 link#6 UHLc 2 3397 - 4 gif0 2001:470:17:7f::2 link#6 UHL10 - 4 lo0
Re: bgpd fails to install ipv6 routes in kernel routing table
Graeme Lee wrote: tico wrote: Graeme Lee wrote: snip Network layout is somewhat complicated. 1 x ebgp and 1 x ibgp session receive ipv4 world tables. Gif tunnel to a hurricane router in Hong Kong. I'm receiving ipv6 world bgp tables from this peer. Connectivity to the peer is fine. Just can't get past it. I can see that my prefix is announced via looking glasses. I'm receiving about 1.6k prefixes from hurricane. I'm speaking BGP over v6 with HE.net as well (albeit in Fremont, not HK), and I can see you just fine, and apparently you can see me (AS30708) as well, since I can ping you from both my Hurricane /64 as well as from an IP within my own /32. $ ping6 -c1 -S 2607:f618:1::1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2607:f618:1::1 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=442.275 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 442.275/442.275/442.275/0.000 ms $ ping6 -c1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2001:470:1:53::2 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=441.775 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 441.775/441.775/441.775/0.000 ms $ bgpctl sho ip bgp 2400:6800::/32 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin *2400:6800::/32 2001:470:1:53::1100 0 6939 10105 i $ uname -mr 4.4 i386 What does your bgpctl sho nex give you? -tico Hi Tico. # bgpctl show next Nexthop State 2001:470:17:7f::1valid gif0UP 203.143.64.133 valid em1 UP, Ethernet, active, 100 MBit/s 121.200.227.93 valid em0 UP, Ethernet, active, 100 MBit/s However, the only reason you can see me is because i've manually stuck in a default route just to get things working # netstat -rnf inet6 Routing tables Internet6: DestinationGateway Flags Refs Use Mtu Prio Iface ::/104 ::1 UGRS 00 - 8 lo0 ::/96 ::1 UGRS 00 - 8 lo0 default2001:470:17:7f::1 UGS0 19 - 8 gif0 ::1::1 UH140 33160 4 lo0 ::127.0.0.0/104::1 UGRS 00 - 8 lo0 ::224.0.0.0/100::1 UGRS 00 - 8 lo0 ::255.0.0.0/104::1 UGRS 00 - 8 lo0 :::0.0.0.0/96 ::1 UGRS 00 - 8 lo0 2001:470:17:7f::/64link#6 UC 10 - 4 gif0 2001:470:17:7f::1 link#6 UHLc 2 3397 - 4 gif0 2001:470:17:7f::2 link#6 UHL10 - 4 lo0 I see. And what do your filters (bgpd, not PF) look like? What changes from a default bgpd.conf have you made? Is there anything peculiar about your gif0 interface? -tico
Re: bgpd fails to install ipv6 routes in kernel routing table
tico wrote:
Graeme Lee wrote:
tico wrote:
Graeme Lee wrote:
snip
Network layout is somewhat complicated. 1 x ebgp and 1 x ibgp
session receive ipv4 world tables. Gif tunnel to a hurricane
router in Hong Kong. I'm receiving ipv6 world bgp tables from this
peer. Connectivity to the peer is fine. Just can't get past it.
I can see that my prefix is announced via looking glasses. I'm
receiving about 1.6k prefixes from hurricane.
I'm speaking BGP over v6 with HE.net as well (albeit in Fremont, not
HK), and I can see you just fine, and apparently you can see me
(AS30708) as well, since I can ping you from both my Hurricane /64
as well as from an IP within my own /32.
$ ping6 -c1 -S 2607:f618:1::1 2001:470:17:7f::2
PING6(56=40+8+8 bytes) 2607:f618:1::1 -- 2001:470:17:7f::2
16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=442.275 ms
--- 2001:470:17:7f::2 ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 442.275/442.275/442.275/0.000 ms
$ ping6 -c1 2001:470:17:7f::2 PING6(56=40+8+8
bytes) 2001:470:1:53::2 -- 2001:470:17:7f::2
16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=441.775 ms
--- 2001:470:17:7f::2 ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 441.775/441.775/441.775/0.000 ms
$ bgpctl sho ip bgp 2400:6800::/32 flags: * = Valid, =
Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete
flags destination gateway lpref med aspath origin
*2400:6800::/32 2001:470:1:53::1100 0 6939 10105 i
$ uname -mr
4.4 i386
What does your bgpctl sho nex give you?
-tico
Hi Tico.
# bgpctl show next
Nexthop State
2001:470:17:7f::1valid gif0UP
203.143.64.133 valid em1 UP, Ethernet, active, 100 MBit/s
121.200.227.93 valid em0 UP, Ethernet, active, 100 MBit/s
However, the only reason you can see me is because i've manually
stuck in a default route just to get things working
# netstat -rnf inet6
Routing tables
Internet6:
DestinationGateway
Flags Refs Use Mtu Prio Iface
::/104 ::1
UGRS 00 - 8 lo0
::/96 ::1
UGRS 00 - 8 lo0
default2001:470:17:7f::1
UGS0 19 - 8 gif0
::1::1
UH140 33160 4 lo0
::127.0.0.0/104::1
UGRS 00 - 8 lo0
::224.0.0.0/100::1
UGRS 00 - 8 lo0
::255.0.0.0/104::1
UGRS 00 - 8 lo0
:::0.0.0.0/96 ::1
UGRS 00 - 8 lo0
2001:470:17:7f::/64link#6
UC 10 - 4 gif0
2001:470:17:7f::1 link#6
UHLc 2 3397 - 4 gif0
2001:470:17:7f::2 link#6
UHL10 - 4 lo0
I see. And what do your filters (bgpd, not PF) look like?
What changes from a default bgpd.conf have you made?
Is there anything peculiar about your gif0 interface?
-tico
There's only one line difference (plus a coment)
allow from any inet6 prefixlen 12 - 64
neighbor 2001:470:17:7f::1 {
remote-as 6939
descr HurricaneHK
local-address 2001:470:17:7f::2
announceIPv4 none
announceIPv6 unicast
set nexthop self
}
# filter out prefixes longer than 24 or shorter than 8 bits
deny from any
allow from any inet prefixlen 8 - 24
# IPv6 Routing
allow from any inet6 prefixlen 12 - 64
# do not accept a default route
deny from any prefix 0.0.0.0/0
# filter bogus networks
deny from any prefix 10.0.0.0/8 prefixlen = 8
deny from any prefix 172.16.0.0/12 prefixlen = 12
deny from any prefix 192.168.0.0/16 prefixlen = 16
deny from any prefix 169.254.0.0/16 prefixlen = 16
deny from any prefix 192.0.2.0/24 prefixlen = 24
deny from any prefix 224.0.0.0/4 prefixlen = 4
deny from any prefix 240.0.0.0/4 prefixlen = 4
# ifconfig gif0
gif0: flags=8051UP,POINTOPOINT,RUNNING,MULTICAST mtu 1280
priority: 0
groups: gif egress
physical address inet 121.200.227.94 -- 216.218.221.2
inet6 fe80::21f:d0ff:fe32:3d58%gif0 - prefixlen 64 scopeid 0x6
inet6 2001:470:17:7f::2 - prefixlen 64
Re: bgpd fails to install ipv6 routes in kernel routing table
Graeme Lee wrote: tico wrote: Graeme Lee wrote: tico wrote: Graeme Lee wrote: snip Network layout is somewhat complicated. 1 x ebgp and 1 x ibgp session receive ipv4 world tables. Gif tunnel to a hurricane router in Hong Kong. I'm receiving ipv6 world bgp tables from this peer. Connectivity to the peer is fine. Just can't get past it. I can see that my prefix is announced via looking glasses. I'm receiving about 1.6k prefixes from hurricane. I'm speaking BGP over v6 with HE.net as well (albeit in Fremont, not HK), and I can see you just fine, and apparently you can see me (AS30708) as well, since I can ping you from both my Hurricane /64 as well as from an IP within my own /32. $ ping6 -c1 -S 2607:f618:1::1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2607:f618:1::1 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=442.275 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 442.275/442.275/442.275/0.000 ms $ ping6 -c1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2001:470:1:53::2 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=441.775 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 441.775/441.775/441.775/0.000 ms $ bgpctl sho ip bgp 2400:6800::/32 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin *2400:6800::/32 2001:470:1:53::1100 0 6939 10105 i $ uname -mr 4.4 i386 What does your bgpctl sho nex give you? -tico Ok forget bgp configs for a minute. I've been quickly scanning over the code, and notable is that the log displays: Feb 9 13:00:15 gw-nextgen bgpd[17223]: send_rtmsg: action 1, prefix 2001:7fb:fe07::/48: Network is unreachable but shouldn't it be a send_rt6msg call in kroute.c?
Re: bgpd fails to install ipv6 routes in kernel routing table
Graeme Lee wrote: Graeme Lee wrote: tico wrote: Graeme Lee wrote: tico wrote: Graeme Lee wrote: snip Network layout is somewhat complicated. 1 x ebgp and 1 x ibgp session receive ipv4 world tables. Gif tunnel to a hurricane router in Hong Kong. I'm receiving ipv6 world bgp tables from this peer. Connectivity to the peer is fine. Just can't get past it. I can see that my prefix is announced via looking glasses. I'm receiving about 1.6k prefixes from hurricane. I'm speaking BGP over v6 with HE.net as well (albeit in Fremont, not HK), and I can see you just fine, and apparently you can see me (AS30708) as well, since I can ping you from both my Hurricane /64 as well as from an IP within my own /32. $ ping6 -c1 -S 2607:f618:1::1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2607:f618:1::1 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=442.275 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 442.275/442.275/442.275/0.000 ms $ ping6 -c1 2001:470:17:7f::2 PING6(56=40+8+8 bytes) 2001:470:1:53::2 -- 2001:470:17:7f::2 16 bytes from 2001:470:17:7f::2, icmp_seq=0 hlim=59 time=441.775 ms --- 2001:470:17:7f::2 ping6 statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 441.775/441.775/441.775/0.000 ms $ bgpctl sho ip bgp 2400:6800::/32 flags: * = Valid, = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin *2400:6800::/32 2001:470:1:53::1100 0 6939 10105 i $ uname -mr 4.4 i386 What does your bgpctl sho nex give you? -tico Ok forget bgp configs for a minute. I've been quickly scanning over the code, and notable is that the log displays: Feb 9 13:00:15 gw-nextgen bgpd[17223]: send_rtmsg: action 1, prefix 2001:7fb:fe07::/48: Network is unreachable but shouldn't it be a send_rt6msg call in kroute.c? On a hunch, I tried a 64bit and a 32 bit machine with 1 prefix each. The 32bit machine adds routes to the kernel without complaint. The 64bit machine complained with send_rtmsg
