ipfw optimizations
Hi, i am looking at (minor) optimizations of the ipfw code in order to reduce the running time in the common cases. I have a few ideas (mostly along the lines of optimizing for the most commonly-used rules). An obvious candidate is the 'match all' rule (all from any to any), but can people suggest other common usage of rules in ipfw ? cheers luigi ---+- Luigi RIZZO, [EMAIL PROTECTED] . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL/FAX: +39-050-568.533/522 . via Diotisalvi 2, 56126 PISA (Italy) Mobile +39-347-0373137 ---+- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
Hi Luigi, i am looking at (minor) optimizations of the ipfw code in order to reduce the running time in the common cases. I have a few ideas (mostly along the lines of optimizing for the most commonly-used rules). An obvious candidate is the 'match all' rule (all from any to any), but can people suggest other common usage of rules in ipfw ? One of the things I would do to optimize ipfw is: - instead of keeping one list with all the rules, split the list (the internal one) by interface and by direction (one list for ed1 incoming, one list for ed1 outgoing, etc.). - then eventually you could be doing the same thing by IP protocol number, but it might not be worth it (with regard to the amount of work required). I think that it is a better way to optimize ipfw than optimize the "match all" rule, since in any security conscious this is likely to be a deny rule, and who cares if it takes a little longer to deny a packet ? My goal usually is to accept legitimate packets as early as possible, reject really obvious stuff also fairly early and then handle the less common stuff. At last there is my match all deny rule, but it does not get exercised that often. One advantage of having a compiled ruleset for each interface would speed up quite a bit the processing by not going over rules that are not applicable. I looked once at doing that on the 3.x-STABLE ipfw, and even if it did not seem to be *too* complicated to do, I did not have the time to go further. Any thoughts ? Patrick. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
One of the things I would do to optimize ipfw is: - instead of keeping one list with all the rules, split the list (the internal one) by interface and by direction (one list for ed1 incoming, one list for ed1 outgoing, etc.). one skipto rule is enough to switch between two rulesets depending on direction, so this is not really worthwhile. I agree that having a `switch' type of rule for selecting interfaces would be a reasonable gain of efficiency (but then again.. how many interfaces is one using!) The problem with current rule format is that you can detect a non-match very early, but in order to have a real match you have to test all the fields (addresses, ports, interfaces, ...) and even if this only means testing flags, it is still some 8-10 tests and 8-10 jumps. cheers luigi ---+- Luigi RIZZO, [EMAIL PROTECTED] . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL/FAX: +39-050-568.533/522 . via Diotisalvi 2, 56126 PISA (Italy) Mobile +39-347-0373137 ---+- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
In message [EMAIL PROTECTED], Luigi Rizzo writes: One of the things I would do to optimize ipfw is: - instead of keeping one list with all the rules, split the list (the internal one) by interface and by direction (one list for ed1 incoming, one list for ed1 outgoing, etc.). one skipto rule is enough to switch between two rulesets depending on direction, so this is not really worthwhile. I agree that having a `switch' type of rule for selecting interfaces would be a reasonable gain of efficiency (but then again.. how many interfaces is one using!) I still think we should split the current "one huge list of rules" into several lists: Two lists per interface: one list of rules for inbound packets one list of rules for outbound packets Two lists for the IP stack: one list of rules for incoming packets one list of rules for outgoing packets One list for forwarding of packets. in theory one could also: Two lists for the UDP stack: one list of rules for incoming packets one list of rules for outgoing packets Two lists for the TCP stack: one list of rules for incoming packets one list of rules for outgoing packets -- Poul-Henning Kamp FreeBSD coreteam member [EMAIL PROTECTED] "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
I still think we should split the current "one huge list of rules" into several lists: Two lists per interface: one list of rules for inbound packets one list of rules for outbound packets Two lists for the IP stack: one list of rules for incoming packets one list of rules for outgoing packets One list for forwarding of packets. aren't these three classes combined in some H-shaped way ? cheers luigi ---+- Luigi RIZZO, [EMAIL PROTECTED] . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL/FAX: +39-050-568.533/522 . via Diotisalvi 2, 56126 PISA (Italy) Mobile +39-347-0373137 ---+- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
In message [EMAIL PROTECTED], Luigi Rizzo writes: I still think we should split the current "one huge list of rules" into several lists: Two lists per interface: one list of rules for inbound packets one list of rules for outbound packets Two lists for the IP stack: one list of rules for incoming packets one list of rules for outgoing packets One list for forwarding of packets. aren't these three classes combined in some H-shaped way ? Could be, the forwarding branch could be a good place to hook up natd(8) for instance... -- Poul-Henning Kamp FreeBSD coreteam member [EMAIL PROTECTED] "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
One of the things I would do to optimize ipfw is: - instead of keeping one list with all the rules, split the list (the internal one) by interface and by direction (one list for ed1 incoming, one list for ed1 outgoing, etc.). one skipto rule is enough to switch between two rulesets depending on direction, so this is not really worthwhile. I agree that having a `switch' type of rule for selecting interfaces would be a reasonable gain of efficiency (but then again.. how many interfaces is one using!) It doesn't matter, it has to do the lookup on a per-interface basis. On my firewall box, I have 11 interfaces. Two ethernet, one loopback, 4 slip, and 4 tunnel. I could easily see a speedup from using per-interface lists. Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
In message [EMAIL PROTECTED], Luigi Rizzo writes: One of the things I would do to optimize ipfw is: - instead of keeping one list with all the rules, split the list (the internal one) by interface and by direction (one list for ed1 incoming, one list for ed1 outgoing, etc.). one skipto rule is enough to switch between two rulesets depending on direction, so this is not really worthwhile. I agree that having a `switch' type of rule for selecting interfaces would be a reasonable gain of efficiency (but then again.. how many interfaces is one using!) I still think we should split the current "one huge list of rules" into several lists: Two lists per interface: one list of rules for inbound packets one list of rules for outbound packets ... I use to think this was the way to do it too, until I went and figured out how to do the exact same thing using the current setup. What we have now is actually more flexiable than this proposed configuration in that it allows a superset of this, plus you don't have to duplicate rules in multiple sets, ie: ipfw add 1000 deny ip from 10.0.0.0/8 to any ipfw add 1001 deny ip from any to 10.0.0.0/8 covers all interfaces, I don't have to add those and the 6 others to every interface rule set like we do on the Ciscos. The skipto situation may be slightly ineffecient due to the number of comparisons needed, perhaps adding the ability to dispatch more directly rather than a chain of skipto's, though I can't come up with a simple syntax for this off the top of my head. :-( -- Rod Grimes - KD7CAX @ CN85sl - (RWG25) [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
Hi, One of the things I would do to optimize ipfw is: - instead of keeping one list with all the rules, split the list (the internal one) by interface and by direction (one list for ed1 incoming, one list for ed1 outgoing, etc.). I often do this manually in long rule sets by using things like ipfw add 1000 skipto 1 from any to any via de0 ipfw add 1001 skipto 2 from any to any via de1 ... ipfw add 1 skipto 15000 from any to any in via de0 #process outbound on de0 rules here ipfw add 15000 blah blah # processing inbound on de0 rules here [...] Anotherwords, don't burden the ipfw with code that can easily be done by an intellegent user, and some more examples/documentation... Yep, and there happens to be a rule that you would like to be tested in every case, but you don't want to test it at the begining (before the switch) but sometime in the middle. With your scheme (which is the only reasonable one currently), you have to duplicate that rule for every branch. This is fine, but if now you need to modify the rule somewhat, don't forget to modify it everywhere. This can rapidly become a maintenance nightmare. What I was proposing is that the per-interface switch be done implicitely by ipfw. So if you do: ipfw add allow ip from joe to bob via de0 ipfw add allow ip from arthur to joe in recv de0 ipfw add allow ip from john to any You get the proper rule tree generated: - ed0 RX: allow ip from joe to bob allow ip from arhur to joe allow ip from john to any - ed0 TX: allow ip from joe to bob allow ip from john to any - ed1 (TX or RX) allow ip from john to any By the way, in terms of optimization you will save: - 2 * number of interfaces rules (the skiptos) that have to be tested for most packets - 2 tests for each rule after (you don't need to retest the interface nor the direction, it has been. If you go further in that logic and implement a per protocol switch, you reduce the number of test even more. To answer a previous question about the number of interfaces, I use FreeBSD as a gateway with 2 ethernet interfaces and 3 tunnel interfaces (ipsec) to remote locations. I guess that most cases where you really worry about ipfw is in gateways where a minimum of 2 interfaces seems reasonable. Again, I am not saying that you can not implement a similar behaviour with ipfw as it is now, I am just saying that if you want to optimize it, you want to reduce the number of test you perform for each rule. What I am proposing is one way of doing it (and as a side effect, it makes managing a tree like set of rule easier). Patrick. matched already) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
On Fri, Jan 07, 2000 at 11:37:02AM -0700, Nate Williams wrote: One of the things I would do to optimize ipfw is: - instead of keeping one list with all the rules, split the list (the internal one) by interface and by direction (one list for ed1 incoming, one list for ed1 outgoing, etc.). one skipto rule is enough to switch between two rulesets depending on direction, so this is not really worthwhile. I agree that having a `switch' type of rule for selecting interfaces would be a reasonable gain of efficiency (but then again.. how many interfaces is one using!) It doesn't matter, it has to do the lookup on a per-interface basis. On my firewall box, I have 11 interfaces. Two ethernet, one loopback, 4 slip, and 4 tunnel. I could easily see a speedup from using per-interface lists. I haven't looked at the firewalling-code in the kernel, but couldn't you gain exactly this speedup by issuing this stuff manually? Add a bunch of "skipto" rules at the very beginning of your ruleset and have them branch to rule 5000, 1, 15000 etc. and then setup your per-interface rules beginning at exactly these rules. In fact, isn't that what Linux' "ipchains" are all about? You split up the rules and branch to one of your rulesets at the beginning. I've never seen anything special in this feature, since ipfw does that as well (you just don't have magical names for your rules but numbers instead). bye, Harold -- Someone should do a study to find out how many human life spans have been lost waiting for NT to reboot. Ken Deboy on Dec 24 1999 in comp.unix.bsd.freebsd.misc To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
I agree that having a `switch' type of rule for selecting interfaces would be a reasonable gain of efficiency (but then again.. how many interfaces is one using!) It doesn't matter, it has to do the lookup on a per-interface basis. On my firewall box, I have 11 interfaces. Two ethernet, one loopback, 4 slip, and 4 tunnel. i meant, if you only have 2-3 interfaces which are used 90% of the times, then you really have 1-2 extra rules to look for. But, in any case, it seems reasonably clear that a 'switch' statement would simplify rule writing in a number of situations, and i agree with Rod that the way ipfw does (having all rules potentially applicable for all cases) is very very flexible and probably more convenient than per-interface lists in many cases. cheers luigi ---+- Luigi RIZZO, [EMAIL PROTECTED] . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL/FAX: +39-050-568.533/522 . via Diotisalvi 2, 56126 PISA (Italy) Mobile +39-347-0373137 ---+- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
(don't you love all that quoting...) I agree that having a `switch' type of rule for selecting interfaces would be a reasonable gain of efficiency (but then again.. how many interfaces is one using!) It doesn't matter, it has to do the lookup on a per-interface basis. On my firewall box, I have 11 interfaces. Two ethernet, one loopback, 4 slip, and 4 tunnel. i meant, if you only have 2-3 interfaces which are used 90% of the times, then you really have 1-2 extra rules to look for. But, in any case, it seems reasonably clear that a 'switch' statement would simplify rule writing in a number of situations, and i agree with Rod that the way ipfw does (having all rules potentially applicable for all cases) is very very flexible and probably more convenient than per-interface lists in many cases. Yes I agree, I love ipfw functionality. You were asking for ideas on how to optimize ipfw. What I suggested is that in its INTERNAL representation of the rules, ipfw could split the rules on a per interface/direction basis. This means that you will not look at the rules that are known to not apply to your interface AND direction, hence saving some time. Again I am not asking for modification of the "user interface" of ipfw which is nice and to the point. As you and Rod mention, the ability to have rules applicable to all interfaces in one shot is great. What I was thinking about is when you build the "per-interface" list of rules, use what is provided in the interface part of the rule to determine where it belongs: ipfw add allow ip from joe to bob in recv ed0 = this rule is to be added only in the inbound list for interface ed0 ipfw add allow ip from joe to bob via ed0 = this rule is to be added to both inbound and outbound lists for i/f ed0 ipfw add allow ip from joe to bob = this rule is to be added to the inbound and outbound lists for all i/fs In the future we could also use negative logic to add a rule to all interfaces except the one mentioned... Also as I said earlier, you don't have to do anymore interface checking when it is implemented this way. The fact that you use such or such list is enough. Also to respond to some comments from Rod, this scheme duplicates the rules, but it does so behind the scene, so it does not add more complexity to the way you configure ipfw. Actually it remains completely compatible with the current behaviour of ipfw. Is that SO unreasonable Patrick. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: ipfw optimizations
No, this is completly reasonable now that I understand what it is your proposing. Even the memory footprint is minimal if pointers to the actual rules is all we store in the per interface list, my largest set duplicated over 8 interfaces would only be 3200 rules. Stored as pointers to rules this would be all of 12.8kbytes, certainly not enough to worry about :-). Good I felt like I was stupid for a while here ;-) Come up with some patches and I'll be glad to test and review them for you... I'll get on that as soon as I am done with that paying job I am doing now :(. Which probably means sometime next week. (don't you hate to have these bills to pay ?) Patrick. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
