On Wed, Jul 12, 2017 at 06:07:28PM +0200, Per-Olov Sjöholm wrote: > Hi > > I have seen net.inet.ip.ifq.drops on my firewall after upgrading the internet > connection and therefor try to tweak it a little. The FW has 4 (but only two > used) physical Intel Gig interfaces. The internal interface has a bunch of > VLANs on it. IPv6 is enabled. > > > I have a linux 8 core Intel atom (C2758 @ 2.40GHz) sitting behind my NAT > OpenBSD 6.0 firewall (CPU N2930 @ 1.83GHz). After increasing > net.inet.ip.ifq.maxlen from default 256 in two steps up to 768 on the > firewall the drops have been less, but still occurs. The performance of the > CentOS 7.3 sitting behind the firewall also have gained approx 150Mbit more > performance in network test against the internet by the > net.inet.ip.ifq.maxlen increase on the OpenBSD firewall. I have now the linux > server sitting behind the fw that gives me the following performance (I have > an 1/1 Gbit fiber in to the house)??? > > [root@server2 tmp]# bbk_cli --live > Start: 2017-07-12 17:35:20 > ISP: Bahnhof Internet AB > Support ID: sth66db38ee9 > Latency: 4.255 ms > Download: 803.603 Mbit/s > Upload: 949.265 Mbit/s > Subscription: 500-1000 Mbit/s fiber > [root@server2 tmp]# > > > And "sysctl -a|grep net.inet.ip.ifq??? now shows... > net.inet.ip.ifq.len=0 > net.inet.ip.ifq.maxlen=768 > net.inet.ip.ifq.drops=1292657 > > > The performance was pretty good even without tweaks :), but is now, as shown > above, 100-150 Mbit better???. But I do have a few questions to you pro:s??? > > # Question > Can I have bad drawbacks by the net.inet.ip.ifq.maxlen increase I have done, > and in what way do I notice it if problem occurs? Or can/should the > net.inet.ip.ifq.maxlen be increased more as I still have drops? Or should I > decrease the value to 512 or to default 256 again do avoid any type of > problem? > Could the net.inet.ip.ifq.drops ideally be zero? Or is that just an ideal > wish that never is true? > Any other parameter to look at at these speeds if I want a well behaved fw > without packet drops and with low latency capable of filling my 1/1 Gbit pipe? >
The size of net.inet.ip.ifq.maxlen should be sized by lookin at your drops pattern. In general if you have bursty traffic you want a bit more. The goal is that the amount of drops are minimal. On our busy firewalls pushing 500Mbps I set maxlen to 4096 and even 8192. Drawback is of a big queue is an increased latency. At the moment having long queues helps when multiple CPUs compete for the big lock since it reduces the packet loss and so TCP suffers less. The latency increase is something we decided to accept since we're not that latency sensitive. The long term plan is actually to get rid of this queue and knob but we're not right there yet. -- :wq Claudio
