I've started seeing the following problem after updating my
carp setup to -current yesterday. But it is probably a bit older.
My carp setup uses IPv6.
The carp master complains as follows:
Oct 1 22:10:19 dougal /bsd: carp: checksum failed, on carp0
The carp slave is also seeing checksum errors, and they cause it to
flip between master and backup states:
Oct 1 22:00:59 cyril /bsd: carp0: state transition: BACKUP -> MASTER
Oct 1 22:01:08 cyril /bsd: carp0: state transition: MASTER -> BACKUP
Oct 1 22:01:08 cyril /bsd: carp: checksum failed, on carp0
Oct 1 22:01:18 cyril /bsd: carp: checksum failed, on carp0
Oct 1 22:01:38 cyril last message repeated 2 times
Oct 1 22:01:39 cyril /bsd: carp0: state transition: BACKUP -> MASTER
Oct 1 22:01:48 cyril /bsd: carp0: state transition: MASTER -> BACKUP
Oct 1 22:01:48 cyril /bsd: carp: checksum failed, on carp0
Oct 1 22:01:58 cyril /bsd: carp: checksum failed, on carp0
carp stats on either side look like this:
$ netstat -s -p carp
carp:
48 packets received (IPv4)
48 packets received (IPv6)
0 packets discarded for bad interface
0 packets discarded for wrong TTL
0 packets shorter than header
48 discarded for bad checksums
0 discarded packets with a bad version
0 discarded because packet too short
0 discarded for bad authentication
0 discarded for unknown vhid
0 discarded because of a bad address list
0 packets sent (IPv4)
0 packets sent (IPv6)
0 send failed due to mbuf memory error
0 transitions to master
The problem is that the IPv6 input path uses IP6_EXTHDR_GET() to
obtain a pointer to the carp header when verifying the carp header's
checksum. IP6_EXTHDR_GET() internally uses m_pulldown(), which might
return a pointer to a different mbuf in the chain. However, there is
no way for the caller of IP6_EXTHDR_GET() to get at the different mbuf
pointer returned by m_pulldown().
The diff below expands the IP6_EXTHDR_GET() code inline and uses
the different mbuf if it is returned.
Maybe there is a simpler way to write this diff, but it has
the desired effect:
$ netstat -s -p carp
carp:
94 packets received (IPv4)
94 packets received (IPv6)
0 packets discarded for bad interface
0 packets discarded for wrong TTL
0 packets shorter than header
0 discarded for bad checksums
0 discarded packets with a bad version
0 discarded because packet too short
0 discarded for bad authentication
0 discarded for unknown vhid
0 discarded because of a bad address list
14 packets sent (IPv4)
14 packets sent (IPv6)
0 send failed due to mbuf memory error
1 transition to master
Note that this diff conflicts with the carp error message diff
I sent earlier. However, this diff is about a real bug so I'll wait for
this to get fixed and then adjust my error message diff as necessary.
ok?
Index: ip_carp.c
===================================================================
RCS file: /cvs/src/sys/netinet/ip_carp.c,v
retrieving revision 1.190
diff -u -p -r1.190 ip_carp.c
--- ip_carp.c 6 Sep 2011 16:00:22 -0000 1.190
+++ ip_carp.c 2 Oct 2011 14:40:34 -0000
@@ -622,6 +622,7 @@ carp6_proto_input(struct mbuf **mp, int
struct ip6_hdr *ip6 = mtod(m, struct ip6_hdr *);
struct carp_header *ch;
u_int len;
+ struct mbuf *m2, *mch;
carpstats.carps_ipackets6++;
@@ -648,9 +649,22 @@ carp6_proto_input(struct mbuf **mp, int
return (IPPROTO_DONE);
}
- /* verify that we have a complete carp packet */
- len = m->m_len;
- IP6_EXTHDR_GET(ch, struct carp_header *, m, *offp, sizeof(*ch));
+ /* verify that we have a complete carp packet and ensure that
+ * the carp header is in a continuous region of memory */
+ len = m->m_pkthdr.len;
+ m2 = NULL;
+ if (m->m_len >= (*offp + sizeof(*ch)))
+ ch = (struct carp_header *)(mtod(m, caddr_t) + *offp);
+ else {
+ m2 = m_pulldown(m, *offp, sizeof(*ch), offp);
+ if (m2) {
+ if (m2->m_len < *offp + sizeof(*ch))
+ panic("m_pulldown malfunction");
+ ch = (struct carp_header *)(mtod(m2, caddr_t) + *offp);
+ } else {
+ ch = NULL;
+ }
+ }
if (ch == NULL) {
carpstats.carps_badlen++;
CARP_LOG(LOG_INFO, sc, ("packet size %u too small", len));
@@ -659,15 +673,16 @@ carp6_proto_input(struct mbuf **mp, int
/* verify the CARP checksum */
- m->m_data += *offp;
- if (carp_cksum(m, sizeof(*ch))) {
+ mch = m2 ? m2 : m;
+ mch->m_data += *offp;
+ if (carp_cksum(mch, sizeof(*ch))) {
carpstats.carps_badsum++;
CARP_LOG(LOG_INFO, sc, ("checksum failed, on %s",
m->m_pkthdr.rcvif->if_xname));
m_freem(m);
return (IPPROTO_DONE);
}
- m->m_data -= *offp;
+ mch->m_data -= *offp;
carp_proto_input_c(m, ch, 1, AF_INET6);
return (IPPROTO_DONE);
