Hi Americo,
(hope the other readers can bear me expounding a bit on this topic; I feel
it's good to understand what one is measuring, when undertaking measurements,
and this is about comparative measurements that should help this understanding.
And maybe we'll learn about an interesting iptables optimization, when the
numbers have been produced)
> >So, if your analysis includes a part where you make a small modification
> >giving a good measurable impact, I'd propose you see how to give two
> >bits saying "check source interface / destination interface" to each rule,
> >and only touch those cachelines when the bits are set.
>
> --- cachelines ?? --- what do you mean?
A cacheline is the unit loaded into processor caches from main memory.
It is 32 byte in size on the P-III. Loading a single cacheline from main
memory, takes about 250 cycles on a 1Ghz P-III. During these cycles, nothing
much else can be done by the processor - it's pure latency "waiting for
main memory".
Thus, if one can systematically save loading a cache line at all, one can
systematically save that latency.
This is THE driving principle behind data structure layout in the kernel
(and in good user level apps, too).
Now, in each individual iptables rule, there is one cacheline holding
the "-i XXX+" interface and mask, and another holding the "-o YYY+"
interface and mask. That's 500 cycles _per_rule_. And they are always
read and checked, even if the user did not use "-i" and "-o" at all!
> You mean rules that look something like this:
>
> iptables -P INPUT ACCEPT
> iptables -A INPUT -i eth0 -s X.X.X.X -j DROP
> (... add more rules to 'filter_eth' chain )
>
> Hold on! doing this will make iptables run through the algorithm twice,
> right?
No.
> - because it would jump to 'ip_packet_match( )',
Yes.
> - run the IP address code first (and find no match),
Yes.
> - then the interface (find the match),
Yes.
> - and then jump again to 'ip_packet_match( )' to find matches for IP addrss
No. How did you come to this conclusion?
> ...so we are going through the code twice, correct? Is that what we are
> looking for?
No, it was not was I was looking for. What I was getting at, is the fact
that in ip_packet_match(), both input and output interface is checked.
No conditional. Not only if the user specified "-i" and "-o", but _always_,
and _for_each_rule_individually_.
My proposal is for you to remove that check (as a test), and see how much
this removal of the interface check improves the results of your baseline
measurements.
> What about just? (example 2)
> iptables -P INPUT DROP
> iptables -A INPUT -i eth0 -j ACCEPT
>
> Is it better if I try it like this? just to test how long that piece of code
> takes?
>
> PS: after that test I would add your patch and test everything over again
> without checking of "source or destination interface", correct!?
Almost. Try it with example 3:
iptables -P INPUT DROP
iptables -A INPUT -s 127.0.0.2 -j ACCEPT
[... repeat last line 1, 5, 10, 20, 50 times...]
If I understood you correctly, you already do the repeating, and, for example,
if you do 20 repeats, and measure latency X, you can say "X/20" is the average
latency for a single rule. Right?
Now, repeat _this_ with the "no interface check" patch I made, and see how much
"X/20" improves.
Americo, I hope I could clear up any confusion I may have caused to you.
If not, it would be best if you printed out my mails, stashed them away,
and forgot about them for the next days, while you continue on your own
plan of measuring.
best regards, and a low variance to your measurements!
Patrick
