as somebody might remember I asked the same question a couple of months ago. Unfortunately I have not received a good reply to the question, then I have started hacking myself on the problem (that's why ntop is somehow in stand-by mode since 10 days or so).
Well the answer is: the problem is livelock (see the enclosed report to see that even Windows is better than Linux/BSD without polling). Due to the way Linux 2.4 works, under high traffic (high with respect to the cpu) the pc drops packets because of interrupt livelock, i.e. the pc spends more time handling interrupts than doing its job. The situation improves under 2.6 (test7) if you use NAPI (New API) that implements polling (i.e. you turn off interrupts as soon as you receive the 1st pkt, then you poll packets and you re-enable interrupts as soon as there are no more packets). Under FreeBSD you can do it this way: http://info.iet.unipi.it/~luigi/polling/. So as of today:
- Linux get 2.6 instable and hope your NIC driver supports NAPI (e.g. Intel cards do from 100 Mbit -> 10 Gbit [http://sourceforge.net/projects/e1000/])
- use *BSD and enable polling
Future. As at high speed polling is probably not enough. That's why I'm hacking a new driver that contains a memory ring directly inside the driver. The ring is then accessed via mmap() from userland (e.g. ntop). As you can see in the report, this new model has some improvements in term of speed. I now have to complete the work then try it at Gigabit speed to see if there's a real advantage or not.
Anyone willing to help me or at least provide me a testfield for testing at Gbit speeds?
Cheers, Luca
report.xls
Description: Binary data
On Marted�, ott 21, 2003, at 03:16 Europe/Rome, David Touitou wrote:
Mike Hunter wrote:
That's extremely lame. I wonder if there's a problem with 4.9...do youBut ntop traffic stats are more and more funny... 2500% lost packets just after launch. And after 2 minutes : Total 321,377 Dropped by the kernel 770,577 [239.77 %]
have any way of testing this on 4.8? You may want to write to a freebsd
mailing list....
I'll try it on another server tomorrow (have to change the cable).
I just made a little and fast test.
Stop ntop and then relaunch it for a few seconds (less than one minute) then stop it again.
As usual, very strange threads number at launch, but the most funny is the difference between web interface and logfile.
Web interface says : Total 8,243 Dropped by the kernel 58,939 [715.02 %]
while ntop logs file says (stopped a bit after the cut/paste from the web interface) :
21/Oct/2003 02:55:55 STATS: 52,173 packets received by filter on fxp1
21/Oct/2003 02:55:55 STATS: 3,672 packets dropped by kernel
21/Oct/2003 02:55:55 STATS: 66,033 packets dropped by ntop
There's definitively something broken.
Web stats and logfile stats do not agree at all and I still don't understand how there can be more dropped packets than received packets (either on the web interface or logfile stats).
I've just noticed ntop only shows ONE thread doing "ps aux" or "top", is that normal (build with ./configure on FreeBSD 4.9-RC).I think that ntop is single-threaded.
It is multi-threaded (compiled as MT) but, as someone else mentionned, there might be an issue with FreeBSD pthread.
Getting it to compile with linuxthread (as mysql does) might be the answer to this issue (but not to the stats inconsistancy).
David.
_______________________________________________ Ntop mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop
------- Luca Deri <[EMAIL PROTECTED]>
