Read the history on this (the FAQ before it was shifted to SVN has it - one of the old .tgz files on SourceForge will have that) -
If that's what it is... It's just a poll, so while it uses the cpu, it's low priority and real work supersedes it. The problem was 'fixed' in ntop for the 4.x series (around FreeBSD 4.7 IIRC). FreeBSD fixed their behavior in the 5.x series. Luca stripped out the ntop fix after 5.x became common. You could always put back in some version of the older patches... -----Burton -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Gary Gatten Sent: Thursday, May 14, 2009 5:08 PM To: [email protected] Cc: [email protected] Subject: Re: [Ntop] 3.3.x threading issue on FreeBSD 6.0 OK, 3.2.1 DOES have similar thread behavior when monitoring with top. When switching from KSEREL to RUN, threads and priority drop as with 3.3.x. The diff is, whatever it's doing during the RUN part must not be NEARLY as intensive. CPU Util DOES go up, but only 50% at most and the duration is much shorter. Now I'm back to the GeoIP, IDLE_PURGE and rrd stuff. ... Can one or all of these cause the "drastic" increase in CPU load? Unless you think otherwise, I'll kill this thread and go back to the other(s). I do wonder what you think of the sched.h error and perhaps a different thread library? -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Gary Gatten Sent: Thursday, May 14, 2009 6:43 PM To: [email protected] Cc: [email protected] Subject: Re: [Ntop] 3.3.x threading issue on FreeBSD 6.0 PS: Still wondering if the error: about sched.h exists but cant be compiled... are influencing this? This error is also present in my 3.2.1 config.log, so maybe not... -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Gary Gatten Sent: Thursday, May 14, 2009 6:36 PM To: [email protected] Cc: [email protected] Subject: Re: [Ntop] 3.3.x threading issue on FreeBSD 6.0 Right this second I have two 3.3.9 instances running. Flow rate is down because it's end of day, but: When the 8 interface instance is in the KSEREL state, top is showing 14 threads with a PRI of 20. However, when it switches to the RUN state, that's when the thread count drops down to 6 - 13 and PRI drops to something > 100 and CPU goes crazy. This all goes back to the posts about 3.3.x not keeping up with the same load 3.2.1 had no problems with. The thread behavior is the latest weird thing I've noticed that I hope is related! Maybe top isn't the best tool to help with this, don't know what else to use other than gdb. I'll try / do whatever you guys need to help me resolve this! The easy answer would prolly be to get a better machine and run some Linux. I like solving problems, but when I'm ALMOST clueless and basically just guessing, I'm just wasting time instead of solving anything! I'm learning more about multithreading and the various schedulers. Which BTW, I can control which scheduler is used on a per binary basis (/etc/libmap.conf), so if there's a particular one you prefer I could look into making that happen. I'm switching back to 3.2.1 now to get more info, but I can't replicate the flow load now so not sure how much it will help. I updated gmake and automake so I'll compile 3.3.9 again with that - also using gcc44. TIA! Gary -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Luca Deri Sent: Thursday, May 14, 2009 6:12 PM To: [email protected] Cc: [email protected] Subject: Re: [Ntop] 3.3.x threading issue on FreeBSD 6.0 Gary ntop definitively spawns threads at the beginning and not dynamically. If your info is correct there's something very broken on ntop+FreeBSD. Please check again Luca On May 15, 2009, at 12:39 AM, Gary Gatten wrote: > <!-- /* Font Definitions */ @font-face {font-family:Tahoma; > panose-1:2 11 6 4 3 5 4 4 2 4;} /* Style Definitions */ p.MsoNormal, > li.MsoNormal, div.MsoNormal {margin:0in; margin-bottom:.0001pt; font- > size:12.0pt; font-family:"Times New Roman";} a:link, > span.MsoHyperlink {color:blue; text-decoration:underline;} > a:visited, span.MsoHyperlinkFollowed {color:purple; text- > decoration:underline;} span.EmailStyle17 {mso-style-type:personal; > font-family:Arial; color:windowtext;} span.EmailStyle18 {mso-style- > type:personal-reply; color:black;} @page Section1 {size:8.5in > 11.0in; margin:1.0in 1.25in 1.0in 1.25in;} div.Section1 > {page:Section1;} --> > Definitely odd thread behavior compared to 3.2.1. > > > > 3.2.1 will spawn whatever threads it needs and the count is fairly > stable as is the priority. 3.3.9 seems to constantly spawn and tear > down threads; sometimes there are 14 (was 20, I decreased the DNSAR > from 9 to 3) and sometimes only 3 or 4! How can this be when I have > 8 interfaces? Nonetheless, when the thread count is high, the > priority is higher (4 - 20) and everything is happy - works great. > Then for some reason the thread count drops, priority drops (>100) > and everything goes to $hit! The cycle repeats over and over. > > > > I've tried hacking Make and configure files, running as root, > running in fg / non-daemon, updated to gcc44 and compiled with > that. Upgraded libtool and several others - nothing seems to be > helping much. > > > > > > > > From: Gary Gatten > Sent: Thursday, May 14, 2009 11:15 AM > To: '[email protected]'; '[email protected]' > Subject: 3.3.x threading issue on FreeBSD 6.0 > > > > Still trying to track down a high load problem since migrating to > 3.3.x - tested 3.3.8 and 3.3.9 and maybe 3.3.1 and 3.3.3, but that > was so long ago I don't recall for sure. > > > > Anyway, on 3.2.1 it starts 20 threads for the nTop instance in > question (8 netflow interfaces) and it operates perfect using > roughly 10%-20% of CPU. 20 threads seem to be constant /stable. > > > > On 3.3.x however, the threads are dynamic from 3 - 20. When the > thread count is high it behaves similar to 3.2.1: the netflow queues > are serviced promptly and cpu load is 10% - 20%. However, for some > reason it kills threads and when there are less than maybe.... 18 > threads, bad things happen: cpu maxes out at 100% and netflow > queues backup and overflow... > > > > Any help would be great. I'll even talk my company into making a > donation - which I was going to do anyway, but will put on the front > burner if I can get some help with this! > > > > TIA! > > > > Gary > > > > "This email is intended to be reviewed by only the intended > recipient and may contain information that is privileged and/or > confidential. If you are not the intended recipient, you are hereby > notified that any review, use, dissemination, disclosure or copying > of this email and its attachments, if any, is strictly prohibited. > If you have received this email in error, please immediately notify > the sender by return email and delete this email from your system." > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop <font size="1"> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'> </div> "This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." </font> _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop <font size="1"> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'> </div> "This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." </font> _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop <font size="1"> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'> </div> "This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." </font> _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
