And .. On my 4.10 machine, it's been running for days now without reaching such CPU load. The CPU speed in that box is about 1/3 of that of my 5.2.1 box, and the network traffic is about ten times that of the 5.2.1 box. So those reports I've seen about traffic/bandwidth usage triggers this, don't make any sense to me either.
I'm sorry if I seem persistent here, or if I was a bit rash in my conclusions earlier, but I'd love to understand this problem better. If this is simply a consequence of the FreeBSD platform, then that is unacceptable and must be fixed (in FreeBSD ofcourse), or I'll have to see if I can dig out someone who can help me come up with a workaround of some kind.
You sure the new threading stuff in 5.x won't be useful?
Thanks, /Eirik
Burton M. Strauss III wrote:
-p is irrelevant - it just allows you to load a different list of protocols. There's no difference in the code.
--set-pcap-nonblocking is not irrelevant? w/o --set-pcap-nonblocking the ntop web server would just appear to hang. With it, processing continues pretty much normally - that's why it's forced on in the ./configure process. (Did you look at the info.html page to see how your options were interpreted??)
Other read herrings - if you read the back traffic, it's not the _atfork() that's the problem, that's just somewhere else FreeBSD is different.
It's the conversion of the libpcap dispatch into a poll that causes high CPU usage. What's undocumented is the way that libpcap uses the bpf object, what's not up for question is that the virtualization of this causes the poll and the apparent hang in 'bpf' state.
There's no cure - it's all implicit in FreeBSD's userland thread model.
-----Burton
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Eirik Oeverby Sent: Friday, July 02, 2004 9:19 AM To: [EMAIL PROTECTED] Subject: Re: [Ntop-dev] High CPU load on FreeBSD with -p option
Hoi,
Burton M. Strauss III wrote:
Lots - did you read docs/FAQ and the back traffic on this list
on this very
topic?
No you didn't... bad user - no candy and cookies for you!
No bad user here, my concience is clean! ;)
I HAVE browsed archives and googled all day, and I know all the stuff you're referring to here. However, if you read my post again, you'll see that the pcap nonblocking option has *no effect* for me. On 4.x, true, it's necessary, but on 5.x, pthreads HAS the _atfork function, and I tried both with and without the nonblock option (and all the other debugging/troubleshooting options) - to no avail.
So I'm sorry, but this seems to be a different issue...
/Eirik
Q. I remember rumors about something not being right under FreeBSD with threads? A. Yes. See FreeBSD bug bin/17437 at http://www.FreeBSD.org/cgi/query-pr.cgi?pr=17437
Basically, due to limits in FreeBSD, there is no pthread_atfork() function. So, when ntop does it's fork() call to create http pages, it
can't fixup
the Mutexes. It wrong and could conceivably cause problems.
However - ntop ran for years without the pthread_atfork()
code, so we're
no worse off in 3.0 under FreeBSD than in 2.2 or 2.1...
There was a flurry of problems late in the 3.0 development
cycle having
to do with a seeming deadlock of the ntop web server (it's actually not dead, just walking at about 0.001KPH).
Thanks to Yeoman efforts by Stanley Hopcroft, Michal Meloun
and, well,
me, we have a work-around.
If you're running under FreeBSD, use the flag,
--set-pcap-nonblocking.
For more on this, read the threads at gmane - look for "FreeBSD and pthreads" - that's probably the best summary. But there's stuff on this back at least to October 2003 - look for Stanley's problems with CPU usage.
-----Burton
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Eirik Oeverby Sent: Friday, July 02, 2004 8:05 AM To: [EMAIL PROTECTED] Subject: [Ntop-dev] High CPU load on FreeBSD with -p option
Hi all,
For some time now I have struggled with ntop on some (not all) of our FreeBSD machines. In particular, a FreeBSD 5.2.1 machine is giving me headaches:
When running ntop (3.0) with the -p <protocol list file> parameter, after a short while its CPU load skyrockets and stays at 100% (actually, 100% of one of the CPUs in the server). Removing the -p parameter normalizes the situation. Note that while the CPU load is high, ntop seems to be behaving as it should, i.e. it collects traffic data and respons to webserver requests. There's no obvious slowdown, though this is a dual opteron server so I don't usually notice slowdowns anyway.
Also note that the --set-pcap-nonblocking option makes no difference; it probably has no significant effect on FreeBSD 5.x.
On another server (FreeBSD 4.10), I have no problems with the -p parameter whatsoever.
Anyone got any clue what's up with this?
Best regards, Eirik �verby Unicore AS Oslo, Norway
_______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
_______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
_______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
_______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
_______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
