https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=288882
Bug ID: 288882
Summary: ipv6 flowlabel handling seems inconsistent and the
documentation is too well-hidden
Product: Base System
Version: 14.2-RELEASE
Hardware: Any
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: [email protected]
Reporter: [email protected]
Hi,
so this all started trying to debug a network issue which was very likely
"multipath something on the way, hashing on (src ip, src port, dst ip, dst
port), and one of the links dropping packets, breaking OpenVPN CI testing".
Totally outside of FreeBSD, just giving context.
When trying to pinpoint this, we identified a pair of (src port, dst port)
where OpenVPN UDP packets reliably got lost, and tried to reproduce with "nc
-u", and those worked just fine. Surprise.
tcpdumping more, it turned out that OpenVPN-generated UDP packets all have a
flowlabel of "0", while nc-generated packets have flowlabels set - and the
problematic path component seems to include flowlabel hashing as well.
Setting net.inet6.ip6.auto_flowlabel=0 made nc-generated packets also have a
flowlabel of "0", thus making the problem reproduceable with "nc", and the
colleague debugging this was happy because he could demonstrate the
infrastructure issue at will.
Now I was asking myself "why are OpenVPN packets not having a flow label? how
is flow label generation controlled, besides that global sysctl?" and I started
reading sources, getting more confused, but I started to suspect that socket
writes caused by write() calls (nc) are handled differently from sendto()
(OpenVPN) - this is because of this code in udp6_usrreq.c::udp6_send():
#if defined(ROUTE_MPATH) || defined(RSS)
if (CALC_FLOWID_OUTBOUND_SENDTO) {
uint32_t hash_type, hash_val;
uint8_t pr;
pr = inp->inp_socket->so_proto->pr_protocol;
hash_val = fib6_calc_packet_hash(laddr, faddr,
inp->inp_lport, fport, pr, &hash_type);
m->m_pkthdr.flowid = hash_val;
M_HASHTYPE_SET(m, hash_type);
}
/* do not use inp flowid */
flags |= IP_NODEFAULTFLOWID;
#endif
so under some conditions that have "SENDTO" in their name, it might set up a
flow label, or not, but in any case this #if clause will stop further
processing from using the "inp flowid" which always gets set.
I have cobbled together a small test program (sendsock.c, will be attached)
which demonstrates the difference. It will send 2 UDP packets to a named
host/IP, one with send() and one with sendto(). Call it like this
gert@fbsd14:~/netcat $ cc -o sendsock sendsock.c && ./sendsock 2001:608:4::7
1234 "bla blu"
and see the difference in tcpdump
13:24:56.893356 IP6 (flowlabel 0x560da, hlim 64, next-header UDP (17) payload
length: 15) 2001:608:0:814::fb00:14.45546 > 2001:608:4::7.1234: [bad udp cksum
0x4f66 -> 0x5ed4!] UDP, length 7
13:24:56.893370 IP6 (hlim 64, next-header UDP (17) payload length: 15)
2001:608:0:814::fb00:14.13523 > 2001:608:4::7.1234: [bad udp cksum 0x4f66 ->
0xdbeb!] UDP, length 7
second packet never gets a flowlabel.
So, after a long preface, my questions
- should there be a difference wrt flowlabel handling between send() and
sendto()?
- if not, please consider this a bug report
- is the flow label stuff documented anywhere? "man ip6" and "man inet6" are
fairly quiet on this. Kernel code suggests that there are setsockopt() otions
to influence flow label handling on a per-socket basis (IPV6_AUTOFLOWLABEL,
IPV6_RECVFLOWID, IPV6_FLOWID) but I could not find anything more
- please consider this a request for easier-to-find documentation
thanks ;-)
--
You are receiving this mail because:
You are the assignee for the bug.
