> On 12 Jul 2017, at 19:25, Claudio Jeker <[email protected]> wrote: > > On Wed, Jul 12, 2017 at 06:07:28PM +0200, Per-Olov Sjöholm wrote: >> Hi >> >> I have seen net.inet.ip.ifq.drops on my firewall after upgrading the >> internet connection and therefor try to tweak it a little. The FW has 4 (but >> only two used) physical Intel Gig interfaces. The internal interface has a >> bunch of VLANs on it. IPv6 is enabled. >> >> >> I have a linux 8 core Intel atom (C2758 @ 2.40GHz) sitting behind my NAT >> OpenBSD 6.0 firewall (CPU N2930 @ 1.83GHz). After increasing >> net.inet.ip.ifq.maxlen from default 256 in two steps up to 768 on the >> firewall the drops have been less, but still occurs. The performance of the >> CentOS 7.3 sitting behind the firewall also have gained approx 150Mbit more >> performance in network test against the internet by the >> net.inet.ip.ifq.maxlen increase on the OpenBSD firewall. I have now the >> linux server sitting behind the fw that gives me the following performance >> (I have an 1/1 Gbit fiber in to the house)??? >> >> [root@server2 tmp]# bbk_cli --live >> Start: 2017-07-12 17:35:20 >> ISP: Bahnhof Internet AB >> Support ID: sth66db38ee9 >> Latency: 4.255 ms >> Download: 803.603 Mbit/s >> Upload: 949.265 Mbit/s >> Subscription: 500-1000 Mbit/s fiber >> [root@server2 tmp]# >> >> >> And "sysctl -a|grep net.inet.ip.ifq??? now shows... >> net.inet.ip.ifq.len=0 >> net.inet.ip.ifq.maxlen=768 >> net.inet.ip.ifq.drops=1292657 >> >> >> The performance was pretty good even without tweaks :), but is now, as shown >> above, 100-150 Mbit better???. But I do have a few questions to you pro:s??? >> >> # Question >> Can I have bad drawbacks by the net.inet.ip.ifq.maxlen increase I have done, >> and in what way do I notice it if problem occurs? Or can/should the >> net.inet.ip.ifq.maxlen be increased more as I still have drops? Or should I >> decrease the value to 512 or to default 256 again do avoid any type of >> problem? >> Could the net.inet.ip.ifq.drops ideally be zero? Or is that just an ideal >> wish that never is true? >> Any other parameter to look at at these speeds if I want a well behaved fw >> without packet drops and with low latency capable of filling my 1/1 Gbit >> pipe? >> > > The size of net.inet.ip.ifq.maxlen should be sized by lookin at your drops > pattern. In general if you have bursty traffic you want a bit more. The > goal is that the amount of drops are minimal. > > On our busy firewalls pushing 500Mbps I set maxlen to 4096 and even 8192. > Drawback is of a big queue is an increased latency. > At the moment having long queues helps when multiple CPUs compete for > the big lock since it reduces the packet loss and so TCP suffers less. > The latency increase is something we decided to accept since we're not > that latency sensitive. > > The long term plan is actually to get rid of this queue and knob but we're > not right there yet. > > -- > :wq Claudio >
Ahh Tnx Claudio I increased net.inet.ip.ifq.maxlen in steps of 256… I had to increase the net.inet.ip.ifq.maxlen 9 times to 2309 for the net.inet.ip.ifq.drops to stop increasing. At a maxlen of 2309 the drops stopped completley. But all values of net.inet.ip.ifq.maxlen higher than 756 did not give any performance boost (well… Not that I could see). At maxlen of 756 and over, the below output represents the average tests very well when testing against the ISP test servers. Yes I love my OpenBSD FW :) :) :) [root@server2 tmp]# bbk_cli --live Start: 2017-07-12 21:18:01 ISP: Bahnhof Internet AB Support ID: sth66e6a1038 Latency: 3.905 ms Download: 858.525 Mbit/s Upload: 970.758 Mbit/s Subscription: 500-1000 Mbit/s fiber [root@server2 tmp]# So.. Is it ideal to raise it to the level where the drops stops even though I see no clear performance gain? If I read you correct, I want to have minimal drops and should therefor in my environment probably use a maxlen of 2309… That also leads to the next question.. I use ADM64 SMP and then we have the big lock discussion…. Can I eventually benefit of changing to uni kernel? I only use PF (with 566 rules), bind for local name resolution and dhcpd. So SMP kernel is probably not needed. But I did not think I could have a penalty using SMP kernel. Can I eventually have a performance penalty using AMD64 SMP instead of AMD64 uni kernel? Of course I want an as low latency as possible :) So… could, theoretically, using the uni instead of SMP kernel eventually give me zero drops at a lower maxlen and therefor lower latency due to removed SMP? I can of course try, but prefer to ask the pros here first :) Peo
