Public bug reported:
Although I've filed this as a docker bug, it is far more likely to be a
kernel (bridge) bug. I have enabled nothing extra in terms of docker
networking. Both of these two
ubuntu72:~/monitor/src/cma $ uname -a
Linux ubuntu72 3.13.0-27-generic #50-Ubuntu SMP Thu May 15 18:06:16 UTC 2014
x86_64 x86_64 x86_64 GNU/Linux
ubuntu72:~/monitor/src/cma $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 14.04 LTS
Release:14.04
Codename: trusty
ubuntu72:~/monitor/src/cma $ brctl show docker0
bridge name bridge id STP enabled interfaces
docker0 8000.d6b7b71c12ab no veth3d9d
vetha7dd
vethade0
I have a couple of docker instances. One is a client with IP address
172.17.0.4, and one a server with address 172.17.0.3.
The client sends out a multicast UDP packet to a certain address
(224.0.2.5) reserved for the use of this software.
The server dutifully listens for the packet, and it receives it. HOWEVER, the
source address of the packet is 172.17.42.1 - which is the address of the
docker interface on the host (and of course, not the proper source address).
The tcpdump trace of this packet being sent is below:
20:28:39.474887 IP 172.17.42.1.bb 224.0.2.5.bb: UDP, length 713
Needless to say, this is extraordinarily confusing to my software - and when it
attempts to reply, nothing good happens...
You can see that it gets 'port unreachable' for that attempt to reply (as it
should).
The subsequent packet sent to the (correct) 0.4 address happens because
that address is in the _content_ of initial multicast packet, and by
then the software is operating on the contents of the packet, not the
(incorrect) origination address..
Below is the full tcpdump trace of what's going on here from the server
perspective:
# tcpdump -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
20:28:35.933507 IP 7db9eb0e89ee igmp.mcast.net: igmp v3 report, 1 group
record(s)
20:28:35.934144 IP 7db9eb0e89ee.42650 10.10.10.20.domain: 18534+ PTR?
22.0.0.224.in-addr.arpa. (41)
20:28:35.960407 IP 10.10.10.20.domain 7db9eb0e89ee.42650: 18534 1/6/0 PTR
igmp.mcast.net. (181)
20:28:35.960737 IP 7db9eb0e89ee.47869 10.10.10.20.domain: 19922+ PTR?
20.10.10.10.in-addr.arpa. (42)
20:28:35.986002 IP 10.10.10.20.domain 7db9eb0e89ee.47869: 19922 0/1/0 (101)
20:28:36.113623 IP 7db9eb0e89ee igmp.mcast.net: igmp v3 report, 1 group
record(s)
20:28:39.474887 IP 172.17.42.1.bb 224.0.2.5.bb: UDP, length 713
20:28:39.475003 IP 7db9eb0e89ee.33748 10.10.10.20.domain: 3617+ PTR?
5.2.0.224.in-addr.arpa. (40)
20:28:39.525434 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091
20:28:39.525488 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb
unreachable, length 556
20:28:39.529629 IP 7db9eb0e89ee.bb 172.17.0.4.bb: UDP, length 924
20:28:39.545780 IP 172.17.0.4.bb 7db9eb0e89ee.bb: UDP, length 70
20:28:39.568680 IP 10.10.10.20.domain 7db9eb0e89ee.33748: 3617 NXDomain 0/1/0
(97)
20:28:39.568878 IP 7db9eb0e89ee.44605 10.10.10.20.domain: 47116+ PTR?
1.42.17.172.in-addr.arpa. (42)
20:28:39.593858 IP 10.10.10.20.domain 7db9eb0e89ee.44605: 47116 0/1/0 (101)
20:28:39.594120 IP 7db9eb0e89ee.36687 10.10.10.20.domain: 20755+ PTR?
4.0.17.172.in-addr.arpa. (41)
20:28:39.620179 IP 10.10.10.20.domain 7db9eb0e89ee.36687: 20755 0/1/0 (100)
20:28:40.941627 ARP, Request who-has 172.17.42.1 tell 7db9eb0e89ee, length 28
20:28:40.941687 ARP, Reply 172.17.42.1 is-at d6:b7:b7:1c:12:ab (oui Unknown),
length 28
20:28:41.638772 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091
20:28:41.638811 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb
unreachable, length 556
20:28:43.639166 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091
20:28:43.639193 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb
unreachable, length 556
On the client side, the packet was sent correctly (according to tcpdump):
20:48:13.277353 IP 172.17.0.4.bb 224.0.2.5.bb: UDP, length 713
So, somebody somewhere is screwing over my source IP addresses...
This could _conceivably_ be considered a security bug I suppose because
information being misrouted might be security-sensitive, and would be
visible to the wrong party.
Below is some misc networking configuration for the host and the two
containers:
ubuntu72:~/monitor/src/cma $ uname -n; ifconfig; route -n
ubuntu72
docker0 Link encap:Ethernet HWaddr d6:b7:b7:1c:12:ab
inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::e8cd:69ff:feb7:d419/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:786010 errors:0 dropped:0 overruns:0 frame:0
TX packets:1180631 errors:0 dropped:0
