Yes, I've got 512MB, as you can see very little in use. I'm using Netflow to collect my data from the gateway router, so I probably get quite a bit more data then you may observer (I get about 50GB of traffic per day, running through my gateway router). I mean does it have to keep the whole database in ram? If so how does something expire from the database, if buying more RAM will allow it to work smoothly that's no problem, I just don't want to buy the RAM and find it all gets chewed up too.
Regards MIKE 47 processes: 46 sleeping, 1 running, 0 zombie, 0 stopped CPU0 states: 0.0% user, 1.0% system, 0.0% nice, 98.1% idle CPU1 states: 0.0% user, 0.0% system, 0.0% nice, 100.0% idle Mem: 513904K av, 92616K used, 421288K free, 0K shrd, 20088K buff Swap: 224868K av, 0K used, 224868K free 38208K cached On Tue, 2002-04-30 at 16:44, Burton M. Strauss III wrote: > Well, there are lots of other places to "use up" memory, esp w/ 32K hosts and > ghu knows how many sessions. > > Seriously, how much memory are we talking about? > > On my network, ntop uses about 8% of 256MB (call it 20-21MB) and that's for > about 6 machines, most inactive, with about 24 hosts contacted... Each host > uses up ~2K, each protocol takes up space in each host entry, etc. (Check > ntop.h for HostTraffic) > > You might #define MEMORY_DEBUG - but I don't know if that version will even > compile > > > -----Burton > > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of > Michael Baird > Sent: Tuesday, April 30, 2002 2:58 PM > To: [EMAIL PROTECTED] > Subject: RE: [Ntop] Ntop 29-4-02 snapshot > > > Yea, I tried that patch, didn't notice anything different, still used up > all the memory on the machine, after 3 hours. > > Regards > MIKE > > On Tue, 2002-04-30 at 06:09, Burton M. Strauss III wrote: > > That message is not - necessarily - indicative of a problem. (code is in > hash.c around 840). > > > > What it's doing is walking the list to find the right entry. > > > > while(list != NULL) { > > if(list->idx == hostIdx) { > > allRight = 1; > > break; > > } else { > > prevList = list; > > list = list->next; > > } > > } > > > > There could be other reasons (?) for this. But I'm suspicious because of the > proximity. > > > > Have you tried the patch? If you ARE getting the out of memory problem, then > that could explain it. If you're not, then the > > problem is going to take another approach to get the data to figure out what's > up... > > > > -----Burton > > > > > > > > -----Original Message----- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of > > Michael Baird > > Sent: Monday, April 29, 2002 11:35 AM > > To: [EMAIL PROTECTED] > > Subject: RE: [Ntop] Ntop 29-4-02 snapshot > > > > > > Burton, I get bunch of these messages as well along the way, maybe this > > could be related. > > > > 9/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32746) failed [host not found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32747) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32748) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32749) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32750) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32751) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32752) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32753) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32754) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32755) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32756) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32757) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32758) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32759) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32760) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32761) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32762) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32763) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32764) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32765) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32766) failed [host not > > found] > > 29/Apr/2002 12:37:01 ERROR: purgeHostIdx(1,32767) failed [host not > > > > Regards > > MIKE > > > > > > On Mon, 2002-04-29 at 12:20, Burton M. Strauss III wrote: > > > First off, -s has been removed as -s <hashsize> > > > > > > It's now a (future) flag to turn off promiscuous mode. That is the code to > set it is in: > > > > > > initialize.c: 980 > myGlobals.disablePromiscuousMode == 1 ? 0 : 1, > > > main.c: 452 myGlobals.disablePromiscuousMode = 1; > > > globals.h: 105 u_char disablePromiscuousMode; > > > > > > But that's all. > > > > > > AFAIK, there is now no limit on hash size growth. The constant is still > defined, but it's not used anymore: > > > > > > ntop.h: 1311 #define MAX_HASH_SIZE 32768 > > > > > > The code that resizes the hash is this: > > > > > > if(!hostFound) { > > > int sz; > > > > > > list->idx = myGlobals.device[actualDeviceId].actualHashSize; > > > if(myGlobals.device[actualDeviceId].actualHashSize < 512) > > > myGlobals.device[actualDeviceId].actualHashSize = 512; > > > else > > > myGlobals.device[actualDeviceId].actualHashSize *= 2; /* > Double */ > > > sz = > myGlobals.device[actualDeviceId].actualHashSize*sizeof(struct hostTraffic*); > > > myGlobals.device[actualDeviceId].hash_hostTraffic = (struct > > > hostTraffic**)realloc(myGlobals.device[actualDeviceId].hash_hostTraffic, > sz); > > > > memset(&myGlobals.device[actualDeviceId].hash_hostTraffic[list->idx], > > > 0, sizeof(struct > hostTraffic*)*(myGlobals.device[actualDeviceId].actualHashSize-list->idx)); > > > traceEvent(TRACE_INFO, "Extending hash size > [newSize=%d][deviceId=%d]", > > > myGlobals.device[actualDeviceId].actualHashSize, > > > actualDeviceId); > > > } else > > > > > > The only limit is available memory. Remember that the realloc will - under > the covers - require enough memory for the new size > > > (rounded to a full page) + the original size (it does a malloc, copy, free > set internally). > > > > > > Now there is a report that the realloc can fail and cause all kinds of > horrible problems (and I agree that should be tested for), > > > but if you have enough ram, you should be able to grow beyond 32K... > > > > > > You can TRY the attached patch. It's under development, and will ONLY > report the error if the realloc fails!! > > > > > > (All greps are from 2.0.99 ntop-02-04-27.tgz) > > > > > > -----Burton > > > > > > > > > -----Original Message----- > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of > > > Michael Baird > > > Sent: Monday, April 29, 2002 10:35 AM > > > To: [EMAIL PROTECTED] > > > Subject: [Ntop] Ntop 29-4-02 snapshot > > > > > > > > > I'm using ntop with the netflow plugin, and it's finally starting to > > > look promising, except for a couple of issues. It now compiles > > > properly, runs properly, and works properly, however it doesn't clear > > > the hash in RAM, my understanding was at 32768 it should clear the hash > > > and start over, is this incorrect, or do I need to run it with some > > > extra arguments, I tried the -S 1 and -S 2 options, neither work with > > > the netflow plugin, -S 0 does. I'm currently using /usr/local/bin/ntop > > > -u root -d as my only arguments, I've tried -s <hashsize> as described > > > in the FAQ but ntop doesn't pay attention to this either. > > > > > > Regards > > > MIKE > > > > > > > > > > > > _______________________________________________ > > > Ntop mailing list > > > [EMAIL PROTECTED] > > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > > > _______________________________________________ > > Ntop mailing list > > [EMAIL PROTECTED] > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > _______________________________________________ > > Ntop mailing list > > [EMAIL PROTECTED] > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > > _______________________________________________ > Ntop mailing list > [EMAIL PROTECTED] > http://listgateway.unipi.it/mailman/listinfo/ntop > > _______________________________________________ > Ntop mailing list > [EMAIL PROTECTED] > http://listgateway.unipi.it/mailman/listinfo/ntop > _______________________________________________ Ntop mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop
