Read the back traffic - anything with freebsd in the titles - it was three way discussion, between myself, Stanley Hopcroft and Meloun Michal.
http://listgateway.unipi.it/pipermail/ntop-dev/2004-January/thread.html http://listgateway.unipi.it/pipermail/ntop-dev/2004-February/thread.html -----Burton > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Behalf Of Eirik Oeverby > Sent: Friday, July 02, 2004 10:21 AM > To: [EMAIL PROTECTED] > Subject: Re: [Ntop-dev] High CPU load on FreeBSD with -p option > > > Hmm.. This is pretty depressing news. What surprises me is that the CPU > load skyrockets all of a sudden, it stays close to zero for a while > (anything from 5 minutes to an hour or two, then suddenly jumps. And it > doesn't climb slowly either, it's almost as if it jumps as a result of a > webpage refresh or something. > > And .. On my 4.10 machine, it's been running for days now without > reaching such CPU load. The CPU speed in that box is about 1/3 of that > of my 5.2.1 box, and the network traffic is about ten times that of the > 5.2.1 box. So those reports I've seen about traffic/bandwidth usage > triggers this, don't make any sense to me either. > > I'm sorry if I seem persistent here, or if I was a bit rash in my > conclusions earlier, but I'd love to understand this problem better. If > this is simply a consequence of the FreeBSD platform, then that is > unacceptable and must be fixed (in FreeBSD ofcourse), or I'll have to > see if I can dig out someone who can help me come up with a workaround > of some kind. > > You sure the new threading stuff in 5.x won't be useful? > > Thanks, > /Eirik > > Burton M. Strauss III wrote: > > -p is irrelevant - it just allows you to load a different list > of protocols. > > There's no difference in the code. > > > > --set-pcap-nonblocking is not irrelevant? w/o > --set-pcap-nonblocking the > > ntop web server would just appear to hang. With it, processing > continues > > pretty much normally - that's why it's forced on in the > ./configure process. > > (Did you look at the info.html page to see how your options were > > interpreted??) > > > > Other read herrings - if you read the back traffic, it's not > the _atfork() > > that's the problem, that's just somewhere else FreeBSD is different. > > > > It's the conversion of the libpcap dispatch into a poll that > causes high CPU > > usage. What's undocumented is the way that libpcap uses the bpf object, > > what's not up for question is that the virtualization of this causes the > > poll and the apparent hang in 'bpf' state. > > > > There's no cure - it's all implicit in FreeBSD's userland thread model. > > > > -----Burton > > > > > >>-----Original Message----- > >>From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > >>Behalf Of Eirik Oeverby > >>Sent: Friday, July 02, 2004 9:19 AM > >>To: [EMAIL PROTECTED] > >>Subject: Re: [Ntop-dev] High CPU load on FreeBSD with -p option > >> > >> > >>Hoi, > >> > >>Burton M. Strauss III wrote: > >> > >>>Lots - did you read docs/FAQ and the back traffic on this list > >> > >>on this very > >> > >>>topic? > >>> > >>>No you didn't... bad user - no candy and cookies for you! > >> > >>No bad user here, my concience is clean! ;) > >> > >>I HAVE browsed archives and googled all day, and I know all the stuff > >>you're referring to here. However, if you read my post again, you'll see > >>that the pcap nonblocking option has *no effect* for me. On 4.x, true, > >>it's necessary, but on 5.x, pthreads HAS the _atfork function, and I > >>tried both with and without the nonblock option (and all the other > >>debugging/troubleshooting options) - to no avail. > >> > >>So I'm sorry, but this seems to be a different issue... > >> > >>/Eirik > >> > >> > >>>Q. I remember rumors about something not being right under FreeBSD with > >>> threads? > >>>A. Yes. See FreeBSD bug bin/17437 at > >>> http://www.FreeBSD.org/cgi/query-pr.cgi?pr=17437 > >>> > >>> Basically, due to limits in FreeBSD, there is no pthread_atfork() > >>>function. > >>> So, when ntop does it's fork() call to create http pages, it > >> > >>can't fixup > >> > >>> the Mutexes. It wrong and could conceivably cause problems. > >>> > >>> However - ntop ran for years without the pthread_atfork() > >> > >>code, so we're > >> > >>> no worse off in 3.0 under FreeBSD than in 2.2 or 2.1... > >>> > >>> There was a flurry of problems late in the 3.0 development > >> > >>cycle having > >> > >>>to > >>> do with a seeming deadlock of the ntop web server (it's actually not > >>>dead, > >>> just walking at about 0.001KPH). > >>> > >>> Thanks to Yeoman efforts by Stanley Hopcroft, Michal Meloun > >> > >>and, well, > >> > >>>me, > >>> we have a work-around. > >>> > >>> If you're running under FreeBSD, use the flag, > >> > >>--set-pcap-nonblocking. > >> > >>> For more on this, read the threads at gmane - look for "FreeBSD and > >>>pthreads" - > >>> that's probably the best summary. But there's stuff on this back at > >>>least to > >>> October 2003 - look for Stanley's problems with CPU usage. > >>> > >>>-----Burton > >>> > >>> > >>> > >>> > >>> > >>>>-----Original Message----- > >>>>From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > >>>>Behalf Of Eirik Oeverby > >>>>Sent: Friday, July 02, 2004 8:05 AM > >>>>To: [EMAIL PROTECTED] > >>>>Subject: [Ntop-dev] High CPU load on FreeBSD with -p option > >>>> > >>>> > >>>>Hi all, > >>>> > >>>>For some time now I have struggled with ntop on some (not all) of our > >>>>FreeBSD machines. In particular, a FreeBSD 5.2.1 machine is giving me > >>>>headaches: > >>>> > >>>>When running ntop (3.0) with the -p <protocol list file> parameter, > >>>>after a short while its CPU load skyrockets and stays at 100% > (actually, > >>>>100% of one of the CPUs in the server). Removing the -p parameter > >>>>normalizes the situation. > >>>>Note that while the CPU load is high, ntop seems to be behaving as it > >>>>should, i.e. it collects traffic data and respons to > webserver requests. > >>>>There's no obvious slowdown, though this is a dual opteron server so I > >>>>don't usually notice slowdowns anyway. > >>>> > >>>>Also note that the --set-pcap-nonblocking option makes no > difference; it > >>>>probably has no significant effect on FreeBSD 5.x. > >>>> > >>>>On another server (FreeBSD 4.10), I have no problems with the -p > >>>>parameter whatsoever. > >>>> > >>>>Anyone got any clue what's up with this? > >>>> > >>>>Best regards, > >>>>Eirik �verby > >>>>Unicore AS > >>>>Oslo, Norway > >>>> > >>>> > >>>>_______________________________________________ > >>>>Ntop-dev mailing list > >>>>[EMAIL PROTECTED] > >>>>http://listgateway.unipi.it/mailman/listinfo/ntop-dev > >>> > >>> > >>>_______________________________________________ > >>>Ntop-dev mailing list > >>>[EMAIL PROTECTED] > >>>http://listgateway.unipi.it/mailman/listinfo/ntop-dev > >>> > >>> > >> > >>_______________________________________________ > >>Ntop-dev mailing list > >>[EMAIL PROTECTED] > >>http://listgateway.unipi.it/mailman/listinfo/ntop-dev > > > > > > _______________________________________________ > > Ntop-dev mailing list > > [EMAIL PROTECTED] > > http://listgateway.unipi.it/mailman/listinfo/ntop-dev > > > > > > _______________________________________________ > Ntop-dev mailing list > [EMAIL PROTECTED] > http://listgateway.unipi.it/mailman/listinfo/ntop-dev _______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
