Re: PF tcp sessions/s rate evaluation
On 2011-08-16, Quentin Aebischer quentin.aebisc...@usherbrooke.ca wrote: Hello everyone, I'm currently a master degree student, and I'd like to benchmark packet filter over the number of tcp sessions per seconds it can handle. So I've got a very basic setup working, consisting of one server running OpenBSD 4.9 with PF (acting as firewall-router), and 2 PC's running Linux, acting respectively as client and webserver (running apache2 for the last). Basically, the client spams standard HTTP requests to the server via the firewall using a basic HTTP injector tool and evaluates the number of sucessful processed requests per seconds. As one can expect, there is an inverse relationship between the number of sessions/s a firewall can sustain and the size of the object of the request. To achieve maximum throughput, you've got to request big size objects (i.e 50KB or more), whereas to achieve maximum sessions rate per second, you've got to make requests with 0 size objects. Prior to this, I've run some tests with a Linux firewall running iptables, and I've come up with an average rate of 11300 sessions/s for 0 size objects (straight up results, no tweaks or improvements made). Moving on to the OpenBSD tests, I only achieved an average rate of 7000 sessions/s for 0 size object (starting up at 8000, slowly decreasing to 7000 - 6500 ...), which is way above the linux/iptables average rate . I then tried to make some tweaks in /etc/sysctl.conf, but no improvement so far. The ruleset I use is the following (copied from the OpenBSD pf tutorial) : set block-policy drop pass out quick pass in on $WAN inet proto tcp port 80 rdr-to $HTTP_SERVER_IP pass in inet proto icmp all pass in on $LAN. So I come here now to know whether you guys have any idea what sort of tweaks I could try to significantly enhance the number of tcp sessions per seconds processed by PF. I'm kind of a PF newbie, so I'm clueless for the moment . Any hints, thoughts or ideas is appreciated ! Make sure your state limits are high enough (set limits states XX; default is 1). Also check sysctl net.inet.ip.ifq; if there are drops then you may want to gradually increase net.inet.ip.ifq.maxlen Note that the ruleset you have shown does not block anything (default if there is *no* matching rule at all is to statelessly pass a packet).
Re: PF tcp sessions/s rate evaluation
There is not much to tweak, performance-wise. OpenBSD avoids such buttons like the plague, and besides: benchmarks should be run with a stock install, which is what 99% of users are going to be doing as well. You can try looking at the output of 'pfctl -si' and see if any of those is increasing a lot, it may give you some more hints. The only thing that jumps to mind is the states limit; if it's getting hit you'll see the memory counter increase. I can't make any suggestion for a good value for 'set limit states' though because you included zero information about the hardware you're testing on. On Tue, Aug 16, 2011 at 02:12:01PM -0400, Quentin Aebischer wrote: Hello everyone, I'm currently a master degree student, and I'd like to benchmark packet filter over the number of tcp sessions per seconds it can handle. So I've got a very basic setup working, consisting of one server running OpenBSD 4.9 with PF (acting as firewall-router), and 2 PC's running Linux, acting respectively as client and webserver (running apache2 for the last). Basically, the client spams standard HTTP requests to the server via the firewall using a basic HTTP injector tool and evaluates the number of sucessful processed requests per seconds. As one can expect, there is an inverse relationship between the number of sessions/s a firewall can sustain and the size of the object of the request. To achieve maximum throughput, you've got to request big size objects (i.e 50KB or more), whereas to achieve maximum sessions rate per second, you've got to make requests with 0 size objects. Prior to this, I've run some tests with a Linux firewall running iptables, and I've come up with an average rate of 11300 sessions/s for 0 size objects (straight up results, no tweaks or improvements made). Moving on to the OpenBSD tests, I only achieved an average rate of 7000 sessions/s for 0 size object (starting up at 8000, slowly decreasing to 7000 - 6500 ...), which is way above the linux/iptables average rate . I then tried to make some tweaks in /etc/sysctl.conf, but no improvement so far. The ruleset I use is the following (copied from the OpenBSD pf tutorial) : set block-policy drop pass out quick pass in on $WAN inet proto tcp port 80 rdr-to $HTTP_SERVER_IP pass in inet proto icmp all pass in on $LAN. So I come here now to know whether you guys have any idea what sort of tweaks I could try to significantly enhance the number of tcp sessions per seconds processed by PF. I'm kind of a PF newbie, so I'm clueless for the moment . Any hints, thoughts or ideas is appreciated ! --
Re: PF tcp sessions/s rate evaluation
Thx for the reply. Well I've already increased the state table size to 15 entries, 1 was not enough (there was up to 7 simultaneous state entries during the test). Hardware wise, I'm using a xeon 2.4 GHz monocore with 1 GB of RAM. Since this server is used as firewall only, I've raised the kernel space memory to up to 90% of total memory. I don't want to make hasty conclusion, so I'll keep searching.. Ryan McBride mcbr...@openbsd.org a C)critB : There is not much to tweak, performance-wise. OpenBSD avoids such buttons like the plague, and besides: benchmarks should be run with a stock install, which is what 99% of users are going to be doing as well. You can try looking at the output of 'pfctl -si' and see if any of those is increasing a lot, it may give you some more hints. The only thing that jumps to mind is the states limit; if it's getting hit you'll see the memory counter increase. I can't make any suggestion for a good value for 'set limit states' though because you included zero information about the hardware you're testing on. On Tue, Aug 16, 2011 at 02:12:01PM -0400, Quentin Aebischer wrote: Hello everyone, I'm currently a master degree student, and I'd like to benchmark packet filter over the number of tcp sessions per seconds it can handle. So I've got a very basic setup working, consisting of one server running OpenBSD 4.9 with PF (acting as firewall-router), and 2 PC's running Linux, acting respectively as client and webserver (running apache2 for the last). Basically, the client spams standard HTTP requests to the server via the firewall using a basic HTTP injector tool and evaluates the number of sucessful processed requests per seconds. As one can expect, there is an inverse relationship between the number of sessions/s a firewall can sustain and the size of the object of the request. To achieve maximum throughput, you've got to request big size objects (i.e 50KB or more), whereas to achieve maximum sessions rate per second, you've got to make requests with 0 size objects. Prior to this, I've run some tests with a Linux firewall running iptables, and I've come up with an average rate of 11300 sessions/s for 0 size objects (straight up results, no tweaks or improvements made). Moving on to the OpenBSD tests, I only achieved an average rate of 7000 sessions/s for 0 size object (starting up at 8000, slowly decreasing to 7000 - 6500 ...), which is way above the linux/iptables average rate . I then tried to make some tweaks in /etc/sysctl.conf, but no improvement so far. The ruleset I use is the following (copied from the OpenBSD pf tutorial) : set block-policy drop pass out quick pass in on $WAN inet proto tcp port 80 rdr-to $HTTP_SERVER_IP pass in inet proto icmp all pass in on $LAN. So I come here now to know whether you guys have any idea what sort of tweaks I could try to significantly enhance the number of tcp sessions per seconds processed by PF. I'm kind of a PF newbie, so I'm clueless for the moment . Any hints, thoughts or ideas is appreciated ! --
Re: PF tcp sessions/s rate evaluation
Just to clarify a bit, I would not be surprised if IPTables performs more quickly than PF in this particular test, for a couple of reasons: - PF uses a red-black tree for the session tracking, while iptables uses a hash table. The red-black tree means performance scales smoothly as the number of sessions increases (and avoids attacks on individual hash buckets), see http://www.benzedrine.cx/pf-paper.html for details, but the insert/delete cost is higher, which is what you're probably seeing here. - PF does full TCP session checking by default (verifying TCP sequence numbers, etc). You could try testing with 'sloppy' state tracking to see if this makes a difference (NOT recommended in production unless you know what you're doing, so please don't report such results in a performance comparison). One problem with the test setup you've described (doing rdr-to for the session to the webserver) is that you can't repeat the tests without the firewall enabled to determine if it's a PF vs. iptables difference or just an OpenBSD vs. Linux difference. Also, if you post a dmesg of the system you're using and tell us what interfaces $WAN and $LAN are, we can check you're not doing tests with some ghastly network cards... I don't know what knob you've twisted to raise the kernel space memory, but such things are not recommended and I would prefer for you to report worse results with a non-tweaked install than report a bunch of settings that shouldn't be touched. In general, the design goals of PF go somewhat in the following order (I'll admit that we have a lot of work to do still on #2 in some areas of PF): 1) Free software 2) Correct, readable code 3) Secure, robust packet filtering 4) Flexible but simple to use 5) Good performance I don't think anyone here cares much if iptables is 'faster' as long as we're ahead on the first 4 items above. We already have enough fast, insecure systems. -- Bruce Schneier On Tue, Aug 16, 2011 at 08:16:02PM -0400, Quentin Aebischer wrote: Thx for the reply. Well I've already increased the state table size to 15 entries, 1 was not enough (there was up to 7 simultaneous state entries during the test). Hardware wise, I'm using a xeon 2.4 GHz monocore with 1 GB of RAM. Since this server is used as firewall only, I've raised the kernel space memory to up to 90% of total memory. I don't want to make hasty conclusion, so I'll keep searching.. Ryan McBride mcbr...@openbsd.org a C)critB : There is not much to tweak, performance-wise. OpenBSD avoids such buttons like the plague, and besides: benchmarks should be run with a stock install, which is what 99% of users are going to be doing as well. You can try looking at the output of 'pfctl -si' and see if any of those is increasing a lot, it may give you some more hints. The only thing that jumps to mind is the states limit; if it's getting hit you'll see the memory counter increase. I can't make any suggestion for a good value for 'set limit states' though because you included zero information about the hardware you're testing on. On Tue, Aug 16, 2011 at 02:12:01PM -0400, Quentin Aebischer wrote: Hello everyone, I'm currently a master degree student, and I'd like to benchmark packet filter over the number of tcp sessions per seconds it can handle. So I've got a very basic setup working, consisting of one server running OpenBSD 4.9 with PF (acting as firewall-router), and 2 PC's running Linux, acting respectively as client and webserver (running apache2 for the last). Basically, the client spams standard HTTP requests to the server via the firewall using a basic HTTP injector tool and evaluates the number of sucessful processed requests per seconds. As one can expect, there is an inverse relationship between the number of sessions/s a firewall can sustain and the size of the object of the request. To achieve maximum throughput, you've got to request big size objects (i.e 50KB or more), whereas to achieve maximum sessions rate per second, you've got to make requests with 0 size objects. Prior to this, I've run some tests with a Linux firewall running iptables, and I've come up with an average rate of 11300 sessions/s for 0 size objects (straight up results, no tweaks or improvements made). Moving on to the OpenBSD tests, I only achieved an average rate of 7000 sessions/s for 0 size object (starting up at 8000, slowly decreasing to 7000 - 6500 ...), which is way above the linux/iptables average rate . I then tried to make some tweaks in /etc/sysctl.conf, but no improvement so far. The ruleset I use is the following (copied from the OpenBSD pf tutorial) : set block-policy drop pass out quick pass in on $WAN inet proto tcp port 80 rdr-to $HTTP_SERVER_IP pass in inet proto icmp all pass in on $LAN. So I come here now to know whether you guys have any idea what sort of tweaks I could
Re: PF tcp sessions/s rate evaluation
Thx for the reply. Well I've already increased the state table size to 15 entries, 1 was not enough (there was up to 7 simultaneous state entries during the test). Hardware wise, I'm using a xeon 2.4 GHz monocore with 1 GB of RAM. Since this server is used as firewall only, I've raised the kernel space memory to up to 90% of total memory. I don't want to make hasty conclusion, so I'll keep searching.. Regardiing this last bit. REALLY! How did you do that? I'd love to know. Whatever you did does not do what you think it does. In fact, I bet it is doing exactly the opposite of what you think it does.
