Hi colleagues,
I'm facing an issue with packet looping(?) when trying to implement
the following topology using OVS:
Here vnet4 and vnet5 are virtual machines, br-ex has an IP address on
host:
root@alpha:~# ovs-vsctl show
Bridge br-ex
Port vnet5
Interface vnet5
Port br-ex
Interface br-ex
type: internal
Port vnet4
Interface vnet4
ovs_version: "2.13.1"
root@alpha:~# ip a
2: enp9s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP
group default qlen 1000
link/ether a8:a1:59:00:32:ed brd ff:ff:ff:ff:ff:ff
inet S.N.A.T/24 scope global enp9s0
valid_lft forever preferred_lft forever
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN group default qlen 1000
link/ether 86:d1:8b:b3:d7:4d brd ff:ff:ff:ff:ff:ff
inet 100.100.0.1/23 brd 100.100.1.255 scope global br-ex
valid_lft forever preferred_lft forever
and I'm redirecting all specified traffic to/from vnet4 through vnet5
(note: when defining rule (1), I use 'in_port=br-ex' which translated
by OVS to 'in_port=LOCAL') :
root@alpha:~# ovs-ofctl dump-flows br-ex
1) table=0, n_packets=1080, n_bytes=79344,
tcp,in_port=LOCAL,nw_src=3.123.231.xxx,tp_src=443
actions=mod_dl_dst:52:54:00:5b:d4:bf,output:vnet5
2) table=0, n_packets=38, n_bytes=2932,
tcp,in_port=vnet4,nw_dst=3.123.231.xxx,tp_dst=443
actions=mod_dl_dst:52:54:00:5b:d4:bf,output:vnet5
3) table=0, n_packets=820695, n_bytes=408963731, priority=0 actions=NORMAL
Now, when I send request to 3.12.231.xxx from vnet4, I expect that
packets in both directions of conversation can be processed by VM
'vnet5' while both sides will continue as if there is nothing in the
middle.
But in fact it seems that OVS loops packets by rule (1) and they never
reach target:
- while on VM 'vnet4' I see only 'forward' packets to 3.123.231.xxx
and no responses:
15:56:04.472226 ip: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP
(6), length 64)
172.29.24.53.62507 > 3.123.231.xxx.443: Flags [S], cksum 0x2203 (correct),
seq 3019516173, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val
1011439203 ecr 0,sackOK,eol], length 0
15:56:05.478382 ip: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP
(6), length 64)
172.29.24.53.62507 > 3.123.231.xxx.443: Flags [S], cksum 0x1e1b (correct),
seq 3019516173, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val
1011440203 ecr 0,sackOK,eol], length 0
15:56:06.480944 ip: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP
(6), length 64)
172.29.24.53.62507 > 3.123.231.xxx.443: Flags [S], cksum 0x1a33 (correct),
seq 3019516173, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val
1011441203 ecr 0,sackOK,eol], length 0
- and on host's external interface I see SYN/SYN-ACK from/to vnet4:
13:58:15.308422 a8:a1:59:00:32:ed > 00:31:46:0d:43:5a, ethertype IPv4 (0x0800),
length 78: (tos 0x0, ttl 61, id 0, offset 0, flags [DF], proto TCP (6), length 64)
S.N.A.T.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x7ce1 (correct), seq
1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011569352
ecr 0,sackOK,eol], length 0
13:58:15.313846 00:31:46:0d:43:5a > a8:a1:59:00:32:ed, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 48, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > S.N.A.T.62509: Flags [S.], cksum 0x7bbb (correct), seq
3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737
ecr 1011569352,nop,wscale 7], length 0
13:58:16.317557 a8:a1:59:00:32:ed > 00:31:46:0d:43:5a, ethertype IPv4 (0x0800),
length 78: (tos 0x0, ttl 61, id 0, offset 0, flags [DF], proto TCP (6), length 64)
S.N.A.T.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x78f9 (correct), seq
1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011570352
ecr 0,sackOK,eol], length 0
13:58:16.322958 00:31:46:0d:43:5a > a8:a1:59:00:32:ed, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 48, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > S.N.A.T.62509: Flags [S.], cksum 0xdbdc (correct), seq
3566290300, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719741746
ecr 1011570352,nop,wscale 7], length 0
- VM 'vnet5' see tons of SYN-ACK responses, which, possibly, are
looped inside OVS:
13:58:15.302960 52:54:00:7a:9e:a4 > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800),
length 78: (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 64)
172.29.24.53.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x5397 (correct),
seq 1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val
1011569352 ecr 0,sackOK,eol], length 0
13:58:15.302992 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800),
length 78: (tos 0x0, ttl 62, id 0, offset 0, flags [DF], proto TCP (6), length 64)
172.29.24.53.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x5397 (correct),
seq 1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val
1011569352 ecr 0,sackOK,eol], length 0
13:58:15.308659 86:d1:8b:b3:d7:4d > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 47, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271
(correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS
val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308710 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 46, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271
(correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS
val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308764 86:d1:8b:b3:d7:4d > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 45, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271
(correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS
val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308769 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 44, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271
(correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS
val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308792 86:d1:8b:b3:d7:4d > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 43, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271
(correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS
val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308794 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 42, id 0, offset 0, flags [DF], proto TCP (6), length 60)
3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271
(correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS
val 3719740737 ecr 1011569352,nop,wscale 7], length 0
[ ... and so on from 3.123.231.xxx 'S.'s ... ]
So, the question is - whether it's possible to catch and redirect
traffic, which comes from internal port, "landed" in host's system?
Thank you.
--
Volodymyr Litovka
"Vision without Execution is Hallucination." -- Thomas Edison
