Hi
[ I did recently migrate my servers from bare metal to cloud instances
(OpenStack at OVH) ]
[ FreeBSD 11.1-STABLE #0 r327055
]
My setup is as follows and didn't change for the last couple of years:
extIF0/pf/NAT <—> epairXa (bridge0) epairXb <-> jail
Downloading a file (by wget) at the host is around 30 MB/s, and an example
tcpdump at extIF0 looks as follows:
19:32:10.711769 IP (tos 0x20, ttl 56, id 37539, offset 0, flags [DF], proto TCP
(6), length 8680)
remote.http > myhost.14367: Flags [.], cksum 0x64ed (incorrect -> 0x3223),
seq 5753:14381, ack 146, win 235, options [nop,nop,TS val 1007145732 ecr
3995852], length 8628: HTTP
19:32:10.713851 IP (tos 0x20, ttl 56, id 37545, offset 0, flags [DF], proto TCP
(6), length 1490)
remote.http > myhost.14367: Flags [.], cksum 0x48d7 (incorrect -> 0x8d1e),
seq 14381:15819, ack 146, win 235, options [nop,nop,TS val 1007145732 ecr
3995852], length 1438: HTTP
19:32:10.713899 IP (tos 0x20, ttl 56, id 37546, offset 0, flags [DF], proto TCP
(6), length 1490)
remote.http > myhost.14367: Flags [.], cksum 0x48d7 (incorrect -> 0x6ade),
seq 15819:17257, ack 146, win 235, options [nop,nop,TS val 1007145732 ecr
3995852], length 1438: HTTP
19:32:10.713934 IP (tos 0x20, ttl 56, id 37547, offset 0, flags [DF], proto TCP
(6), length 1490)
remote.http > myhost.14367: Flags [.], cksum 0x48d7 (incorrect -> 0x1173),
seq 17257:18695, ack 146, win 235, options [nop,nop,TS val 1007145732 ecr
3995852], length 1438: HTTP
19:32:10.713962 IP (tos 0x20, ttl 56, id 37548, offset 0, flags [DF], proto TCP
(6), length 1490)
remote.http > myhost.14367: Flags [.], cksum 0x48d7 (incorrect -> 0xcf7a),
seq 18695:20133, ack 146, win 235, options [nop,nop,TS val 1007145732 ecr
3995852], length 1438: HTTP
When downloading the very same file within a VIMAGE jail the performance drops
to around 80 KB/s, quite a dramatic loss. An example tcpdump at exitIF0 looks
as follows:
19:34:36.284175 IP (tos 0x0, ttl 56, id 28618, offset 0, flags [DF], proto TCP
(6), length 2948)
remote.http > myhost.63382: Flags [.], cksum 0x5df6 (incorrect -> 0x4478),
seq 1449:4345, ack 146, win 235, options [nop,nop,TS val 1007182125 ecr
4141429], length 2896: HTTP
19:34:36.481904 IP (tos 0x0, ttl 56, id 28620, offset 0, flags [DF], proto TCP
(6), length 1500)
remote.http > myhost.63382: Flags [.], cksum 0xd11d (correct), seq
1449:2897, ack 146, win 235, options [nop,nop,TS val 1007182175 ecr 4141429],
length 1448: HTTP
19:34:36.484109 IP (tos 0x0, ttl 56, id 28621, offset 0, flags [DF], proto TCP
(6), length 2948)
remote.http > myhost.63382: Flags [.], cksum 0x5df6 (incorrect -> 0x2e5b),
seq 15929:18825, ack 146, win 235, options [nop,nop,TS val 1007182175 ecr
4141629], length 2896: HTTP
19:34:36.682006 IP (tos 0x0, ttl 56, id 28623, offset 0, flags [DF], proto TCP
(6), length 1500)
remote.http > myhost.63382: Flags [.], cksum 0x4ab6 (correct), seq
2897:4345, ack 146, win 235, options [nop,nop,TS val 1007182225 ecr 4141629],
length 1448: HTTP
19:34:36.684159 IP (tos 0x0, ttl 56, id 28624, offset 0, flags [DF], proto TCP
(6), length 2948)
remote.http > myhost.63382: Flags [.], cksum 0x5df6 (incorrect -> 0xd7db),
seq 18825:21721, ack 146, win 235, options [nop,nop,TS val 1007182225 ecr
4141829], length 2896: HTTP
A tcpdump at epairXa looks comparable.
I did reduce all MTU settings at the involved interfaces from their initial
settings (1490) to an experimental setting of 1400, just to be on the save
side, to no avail. (FYI: I did have to reduce from 1500 to 1490 to please IPSec
after migration from bare metal to cloud infrastructure.)
Then, I did test the following settings found in the Net, to no avail either:
sysctl net.inet.tcp.tso=0
sysctl net.link.bridge.pfil_onlyip=0
sysctl net.link.bridge.pfil_bridge=0
sysctl net.link.bridge.pfil_member=0
sysctl net.add_addr_allfibs=0
I do have to admit that I am lost here, and that I cannot think about what is
going wrong. The last download I did try at my old severs has been some weeks
ago. Ever since I did upgrade FreeBSD 11.1-STABLE, and I did move my
infrastructure from bare metal to cloud, thus I cannot test anymore if my old
servers would have shown that performance issue in the meantime.
Thus any feedback is highly recommended!
Thanks in advance and regards,
Michael
_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[email protected]"
