Hi Burton, Time for some results. (Note: I didn't put your last modification in as it was just cosmetic). Anyway, it looks like the patch is doing its job ok. Ntop indeed stays responsive while updating RRDs in contrary to the situation when there is no forking. However, in my situation, it is still not usable: Jun 16 20:33:34 ntop ntop[1886]: [MSGID0797160] RRD: 37183 updated in 84.789 seconds Jun 16 20:33:34 ntop ntop[1873]: [MSGID8475868] RRD: fork()ed child No child processes(10)
Jun 16 20:47:52 ntop ntop[1933]: [MSGID0797160] RRD: 269383 updated in 642.530 seconds Jun 16 20:47:52 ntop ntop[1873]: [MSGID8475868] RRD: fork()ed child No child processes(10) Jun 16 21:11:31 ntop ntop[1962]: [MSGID0797160] RRD: 355828 updated in 1162.019 seconds Jun 16 21:11:32 ntop ntop[1873]: [MSGID8475868] RRD: fork()ed child No child processes(10) Jun 16 21:32:57 ntop ntop[2006]: [MSGID0797160] RRD: 321247 updated in 1247.692 seconds Jun 16 21:32:57 ntop ntop[1873]: [MSGID8475868] RRD: fork()ed child No child processes(10) As you can see, updating all RRDs in my situation stil takes about 20 minutes. So the hardware is still bottleneck here. (Although this is a setup with the /var directory on a RAID 1+0 array on 4 Ultra160 SCSI disks). I might do some more tweaking, or ditch the RAID1. But a *real* setup would send the data to a seperate database-server anyway. Still, I'd vote for inclusion of the patch since it makes ntop more responsive while doing RRD updates (and of course the forking can be disabled in the web interface). Regards, Robbert ________________________________ Van: Burton M. Strauss III [mailto:[EMAIL PROTECTED] Verzonden: wo 16-6-2004 16:22 Aan: Kouprie, Robbert Onderwerp: RE: [Ntop-dev] Spreading ntop's RRD dumps to spread system load No child processes(10) - is from this: } else if(childpid) { ... pid = waitpid(childpid, &status, 0); if(pid == 0) traceEvent(CONST_TRACE_NOISY, "RRD: fork()ed child no info"); else if(pid < 0) traceEvent(CONST_TRACE_NOISY, "RRD: fork()ed child %s(%d)", strerror(errno), errno); ... After the fork() call, the child goes off to process the RRDs and the parent waits for the child to end. However which process ACTUALLY runs first is indeterminate, and so it seems like the child has run to completion and exited before the waitpid() call (which waits for the child to exit) is executed. Thus you are getting an 'error'. But in fact we should probably just test for ECHILD and ignore it... Make the block look like this if(pid == 0) traceEvent(CONST_TRACE_NOISY, "RRD: fork()ed child no info"); #ifdef RRD_DEBUG else if((pid < 0) && (errno == ECHILD)) traceEvent(CONST_TRACE_NOISY, "RRD_DEBUG: fork()ed child ended normally and first"); #endif else if(pid < 0) traceEvent(CONST_TRACE_NOISY, "RRD: fork()ed child %s(%d)", strerror(errno), errno); else if((WIFEXITED(status) != 0) && (rc = WEXITSTATUS(status) != 0)) traceEvent(CONST_TRACE_NOISY, "RRD: fork()ed child RC %d", rc); #ifdef RRD_DEBUG else traceEvent(CONST_TRACE_NOISY, "RRD_DEBUG: fork()ed child ended normally"); #endif Or just don't worry about it... -----Burton > -----Original Message----- > From: Kouprie, Robbert [mailto:[EMAIL PROTECTED] > Sent: Wednesday, June 16, 2004 6:54 AM > To: Burton M. Strauss III > Subject: RE: [Ntop-dev] Spreading ntop's RRD dumps to spread system load > > > Hi Burton, > > The new patch does this (I enabled the config option): > > Jun 16 13:48:08 ntop ntop[19189]: [MSGID0529826] RRD: Welcome to > the RRD plugin > Jun 16 13:48:08 ntop ntop[19189]: [MSGID0974000] RRD: Mask for > new directories is 0700 > Jun 16 13:48:08 ntop ntop[19189]: [MSGID9274544] RRD: Mask for > new files is 0066 > Jun 16 13:48:08 ntop ntop[19189]: [MSGID8734864] RRD: Will > fork() for reporting > Jun 16 13:48:08 ntop ntop[19189]: [MSGID0704862] RRD: Started > thread (-552649808) for data collection. > Jun 16 13:48:08 ntop ntop[19189]: [MSGID8606478] THREADMGMT: rrd > thread (-552649808) started > Jun 16 13:48:23 ntop ntop[19207]: [MSGID0797160] RRD: 839 > updated in 0.382 seconds > Jun 16 13:48:23 ntop ntop[19189]: [MSGID8475868] RRD: fork()ed > child No child processes(10) > > (...) > > Jun 16 13:51:39 ntop ntop[19249]: [MSGID0797160] RRD: 913 > updated in 0.649 seconds > Jun 16 13:51:39 ntop ntop[19189]: [MSGID8475868] RRD: fork()ed > child No child processes(10) > > What does this "No child processes(10)" message mean exactly? > > -- Robbert > > > -----Oorspronkelijk bericht----- > > Van: Burton M. Strauss III [mailto:[EMAIL PROTECTED] > > Verzonden: dinsdag 15 juni 2004 15:58 > > Aan: Kouprie, Robbert > > Onderwerp: RE: [Ntop-dev] Spreading ntop's RRD dumps to spread system > > load > > > > > > Sorry about that - it was an early test version. It's > > actually work I'm > > doing for a client, but as it's in the GPLed side, it's got > > to be ok for me > > to release it - just don't want to advertise it until I'm > > ready. So please > > don't share the patch yet. > > > > Attached is the updated patch against 3.0. It should apply > > against 3.0.50, > > but I haven't tried it. > > > > Note that this doesn't spread the load, it just de-couples > > the RRDs from > > packet processing - this prevents packet loss, which is what > > I'm working on. > > > > I still think your idea about 'spreading' load is silly - > > just let the OS > > handle it. With this patch you now have an additional ready > > thread that the > > OS can balance against available resources. > > > > However, I *think* you can accomplish what you want by adding a small > > sched_yield() and nanosleep() at the end of the updateRRD() routine: > > > > > > #ifdef CFG_MULTITHREADED > > releaseMutex(&rrdMutex); > > #ifdef MAKE_WITH_SCHED_YIELD > > sched_yield(); /* Allow other threads to run */ > > #endif > > { > > struct timespec nInterval; > > nInterval.tv_sec = 0; > > nInterval.tv_nsec = 3000000; /* 3ms */ > > nanosleep(&nInterval, NULL); > > } > > #endif > > } > > > > That's off the top of my head - may need some work. And read > > the note in > > man nanosleep about small intervals - you don't want it so > > small it gets > > into the busy wait zone. > > > > > > -----Burton > > > > > > > > > -----Original Message----- > > > From: Kouprie, Robbert [mailto:[EMAIL PROTECTED] > > > Sent: Tuesday, June 15, 2004 7:54 AM > > > To: Burton M. Strauss III > > > Cc: [EMAIL PROTECTED] > > > Subject: RE: [Ntop-dev] Spreading ntop's RRD dumps to > > spread system load > > > > > > > > > Hi Burton, > > > > > > This patch makes the RRD thread die when doing the RRD updates: > > > > > > Jun 12 16:04:41 ntop ntop[20333]: [MSGID0067366] Starting > > 'rrdPlugin' > > > Jun 12 16:04:41 ntop ntop[20333]: [MSGID0529826] RRD: Welcome to > > > the RRD plugin > > > Jun 12 16:04:41 ntop ntop[20333]: [MSGID0974000] RRD: Mask for > > > new directories is 0700 > > > Jun 12 16:04:41 ntop ntop[20333]: [MSGID9274544] RRD: Mask for > > > new files is 0066 > > > Jun 12 16:04:41 ntop ntop[20333]: [MSGID0704862] RRD: Started > > > thread (-1140851792) for data collection. > > > > > > (...) > > > > > > Jun 12 16:04:55 ntop ntop[20375]: [MSGID9089762] RRD: FORK > > starting child > > > Jun 12 16:04:55 ntop ntop[20333]: [MSGID0940042] RRD: FORK > > ending parent > > > Jun 12 16:04:55 ntop ntop[20333]: [MSGID0554110] **WARNING** > > > THREADMGMT: rrd thread (-1140851792) terminated > > > Jun 12 16:04:55 ntop ntop[20375]: [MSGID0889950] RRD: FORK > > > dumpMatrix test > > > Jun 12 16:04:55 ntop ntop[20375]: [MSGID0346494] RRD: FORK > > > ending child, 318 RRDs updated > > > > > > Any other suggestions? ;) > > > > > > Regards, > > > Robbert > > > > > > > -----Oorspronkelijk bericht----- > > > > Van: Burton M. Strauss III [mailto:[EMAIL PROTECTED] > > > > Verzonden: donderdag 10 juni 2004 17:26 > > > > Aan: [EMAIL PROTECTED] > > > > CC: Kouprie, Robbert > > > > Onderwerp: RE: [Ntop-dev] Spreading ntop's RRD dumps to > > spread system > > > > load > > > > > > > > > > > > Interesting... > > > > > > > > That's actually easy to do in SOME situations, namely > > those that do > > > > copy-on-write on a fork() call. The fork()ed child gets a > > > > copy of memory as > > > > of the fork() call and can happily do what it wants with it - > > > > subsequent > > > > updates won't impact it. > > > > > > > > I haven't done a lot of testing, but give the enclosed patch > > > > (vs. 3.0) a > > > > try. > > > > > > > > Unfortunately that won't run in all OSes. > > > > > > > > For the others, I guess the question is, can you live with > > > > the RRDs being > > > > out of time-sync? > > > > > > > > -----Burton > > > > > > > > > -----Original Message----- > > > > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] > > > > > Behalf Of Kouprie, Robbert > > > > > Sent: Thursday, June 10, 2004 7:50 AM > > > > > To: [EMAIL PROTECTED] > > > > > Cc: [EMAIL PROTECTED] > > > > > Subject: [Ntop-dev] Spreading ntop's RRD dumps to > > spread system load > > > > > > > > > > > > > > > Hi Luca, ntop-dev, > > > > > > > > > > (Luca, we talked a little bit about this already - so this just > > > > > serves as a reminder and to inform ntop-dev). > > > > > > > > > > Just an idea for something I stumbled upon with the RRD dump > > > > > option in ntop. When RRDdumping to a large number of different > > > > > RRDs the system load gets a big spike. For example, > > when enabling > > > > > host RRD dumps in our setup (on a /16 net with about 25k active > > > > > hosts) it has to update *a lot* of rrds. Although I use a RAID10 > > > > > setup on SCSI disks, still the system gets a high load > > spike when > > > > > it's dumping time. > > > > > > > > > > Maybe it is an idea to spead the total disk I/O (i.e. the RRD > > > > > dumps) within the interval that is set to do the dumps. (So very > > > > > simply put: why do 25000 RRD updates once in 5 minutes, instead > > > > > of 5000 each minute in this 5 minute interval?). > > > > > > > > > > Regards, > > > > > Robbert > > > > > > > > > > >
_______________________________________________ Ntop-dev mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
