Maybe you should have a look on the tcp window size values.
I had such a behavior with two gateways who was not playing the same way
with the TCP window size which made my network damn slow.
If you have a linux gateway in front of your FreeBSD VM, it should be this.

sysctl net.inet.tcp.rfc1323=1 Activate window scaling and timestamp options
according to RFC 1323.sysctl ipc.maxsockbuf=*[sbmax]* Set maximum size of
TCP window.sysctl net.inet.tcp.recvspace=*[wstd]* Set default size of TCP
receive window.sysctl net.inet.tcp.sendspace=*[wstd]* Set default size of
TCP transmit window.sysctl kern.ipc.nmbclusters View maximum number of mbuf
clusters. Used for storage of data packets to/from the network interface.
Can only be set att boot time - see above. sysctl net.inet.tcp.liondmask=7Used
to enable the "netlion" patch above.

*Guillaume Seigneuret*

*Network and System Security Architect*
 *Web* :
 *Address* :
Hôtel Technologique Marseille Innovation - BP 100
45 rue Frederic Joliot Curie
Technopôle de Château Gombert
13382 Marseille Cedex 13

2011/5/12 Laurent Cligny <>

> Hello all,
> I want to setup a FreeBSD DomU VM as a network NAT gateway in order to
> provide Internet access to other FreeBSD and Linux DomU VMs.
> My Dom0 is Xen Cloud Platform 1.0 on a Dell Poweredge 210 with 8 cores,
> 16GiB RAM and one NIC.
> All FreeBSD VM are 8.2 amd64 with XENHVM kernel anf the Linux VM is a
> Paravirtualized Debian amd64.
> The NAT gateway is a two-NIC box, with one (xn0) configured with a
> public Internet address and the other (xn1) configured with a class A
> private address (
> The Internet traffic is very good from my FreeBSD NAT gateway to the
> Internet (~ 50MiB/s) and also between other VMs on the private network
> (~ 70MiB/s trough scp) after applying the patch here
> (
> fixing the "too many frags" problem.
> Here is a schema of my setup with the observed net speed on it:
> ----------         ---------------         ----------------------
> |Internet|---xn0---|FBSD DomU NAT|---xn1---|FBSD and Linux DomUs|
> ----------         ---------------         ----------------------
>          <------->               <------->
>           50MiB/s                 70MiB/s
>          <------------------------------->
>           0KiB/s <network speed< 5 KiB/s
> I tried doing NAT on the FBSD gateway with PF and also with NATd after
> shutting down PF and unloaded the kernel modules pf.ko and pflog.ko, but
> I have the same perf problems with both solutions. Also I tried
> disabling net.inet.tcp.tso and did ifconfig -tso on all NICs on NAT
> gateway and other VMs without change.
> I don't see anything on the error console or in /var/log/message.
> When i'm trying going though NAT with a Linux paravirt DomU, the network
> perf is slightly better but don't go over 5KiB/s.
> Finally I put my Linux paravirt DomU VM as a NAT gateway in place of the
> FreeBSD one, and the network perf is now normal through it (50MiB/s)
> from the FreeBSD DomU VMs.
> As I'm very happy with FreeBSD for my environment I'd like to keep as
> much as this lovely OS for all my needs, so if anyone have an idea where
> the problem can lies, and what to test next to have a clue of what's
> happening, I will be very thankfull.
> Thanks in advance.
> --
> /Regards,/
> *Laurent Cligny
> /Founder/*
> 02 97 37 71 14
> 06 25 40 86 48
> Steadinet <>
> _______________________________________________
> mailing list
> To unsubscribe, send any mail to ""
_______________________________________________ mailing list
To unsubscribe, send any mail to ""

Reply via email to