Sounds like you've got a good handle on your project, resources and capabilities.
Have Fun! -----Burton > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of > pfeito > Sent: Wednesday, April 28, 2004 1:10 PM > To: [EMAIL PROTECTED] > Subject: RE: [Ntop] Post processing of tcpdump files with NTOP > > > Hello, > > A little of context: > I'm doing a simple network flow analysis based on a few days of > traffic. The > overall duration of the collecting process will be no more than 5 > to 7 days. > > > Because this is a limited time experience, I think I prefer to do a post > processing of tcpdump files, that way avoiding to do some > filtering/optimization to what traffic gets processed, as it would not > reflect 100% the traffic on the network. > For continued analysis this method would be impossible to achieve > and, in my > opinion, kind of stupid. > > More memory, would be the best solution of course :) But not > feasible in the > few days I have left. I already collected a day or so of data, by Sunday I > will have all data collected. I will then Zip the files and upload them to > the 1GB ram machine. The uploading will be done after all data is > collected, > thus not affecting bandwidth usage. > > A couple of days ago, I read a few things about nProbe that caught my > attention, but because it isn't free, I chose to use Ntop instead. > > Thanks Burton! > -pfeito > > > > -----Original Message----- > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On > Behalf Of Burton > > M. Strauss III > > Sent: quarta-feira, 28 de Abril de 2004 14:43 > > To: [EMAIL PROTECTED] > > Subject: RE: [Ntop] Post processing of tcpdump files with NTOP > > > > Three choices... > > > > 1. Use the -x and/or -X options to limit the number of hosts ntop > > processes > > to what fits in memory. Crude, but maybe workable. > > > > 2. Use filtering to limit it to the important hosts and/or some of the > > workload reduction options - man ntop. Better than #1, but takes more > > knowledge of your environment. > > > > 3. Buy more memory. 512MB DDR is still under US$70 if you shop > carefully. > > Two Saturday's ago BestBuy ad, for example, PC2700 DDR 512MB US$90 less > > US$35 Mail-in-rebate. > > > > 4. Use a netFlow collector (nProbe, etc.) on the local machine > sending the > > data to the remote. > > > > 5. Use the capture files - but isn't the transport of them causing more > > bandwidth usage??? > > > > OK, that's 5, but ... you get the drift. > > > > > > -----Burton > > > > > > > -----Original Message----- > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of > > > pfeito > > > Sent: Wednesday, April 28, 2004 7:54 AM > > > To: [EMAIL PROTECTED] > > > Subject: RE: [Ntop] Post processing of tcpdump files with NTOP > > > > > > > > > > > > Hello again, > > > > > > > -----Original Message----- > > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On > > > Behalf Of Burton > > > > M. Strauss III > > > > Sent: quarta-feira, 28 de Abril de 2004 4:21 > > > > To: [EMAIL PROTECTED] > > > > Subject: RE: [Ntop] Post processing of tcpdump files with NTOP > > > > > > > > Which version of ntop? 3.0 is MUCH more stable than the 2.2 > > > series. And > > > > should have no problems. > > > > > > Im using 3.0. > > > > > > > > > > > You are wrong about memory usage - ntop doesn't benefit from running > > > > off-line, in fact will probably need more memory because it > > > won't be able > > > > to > > > > purge inactive hosts. > > > > > > I believe so, but in this specific case, the data collecting > PC has only > > > 256MB Ram which becomes exausted after +- 10H when using NTOP in real- > > time > > > processing mode (due to the large amounts of traffic in the network). > > > > > > I also have a remote machine with 1GB ram which I can use, but > > > not connected > > > to the target network, therefore I can only use it to process previous > > > collected data. It takes more memory, but with 1GB I could > > > process more than > > > than 10H. > > > > > > Ideally, it would be better if I could deploy the 1GB ram > machine in the > > > target network and use NTOP in real-time, but this is not the case, > > > unfortunely. > > > > > > > > > > > Certainly the data you're looking for is in the rrd files - you may > > need > > > > to > > > > create some custom graphs using rrdtool, but the data is there. > > > > > > That is interesting. I've to research further on that. I don't have a > > clue > > > how can I make custom graphs with rrdtool, since I dont know > much about > > > rrdtool, only that is used to collect periodically. > > > > > > Thanks for the feedback :) > > > -pfeito > > > > > > > > > > > -----Burton > > > > > > > > > > > > > -----Original Message----- > > > > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of > > > > > pfeito > > > > > Sent: Tuesday, April 27, 2004 7:28 PM > > > > > To: [EMAIL PROTECTED] > > > > > Subject: [Ntop] Post processing of tcpdump files with NTOP > > > > > > > > > > > > > > > Hi to all! > > > > > > > > > > > > > > > > > > > > I'm trying to do some network analysis in a university department > > > > network > > > > > and I choose to use NTOP to acquire statistic data. 10 hours > > > later NTOP > > > > > crashed due to lack of memory (only 256MB were available > in the NTOP > > > > > machine). > > > > > > > > > > > > > > > > > > > > I spent a 2 or 3 hours reading some references, trying to > > > understand the > > > > > memory limitations of NTOP, and, if I understood well, its is kind > > of > > > > > difficult to do a long run analysis (e.g. 1 week or +) with NTOP > > when > > > > > dealing with medium size to large networks, although it really > > depends > > > > on > > > > > the machine specs. > > > > > > > > > > > > > > > > > > > > I decided to try a different approach: to collect raw tcpdump > > > output for > > > > a > > > > > week, and then feed that data to NTOP. I've done a little > > > > > experiment with an > > > > > 1 minute tcpdump file and it seem to work well. > > > > > > > > > > > > > > > > > > > > Will this method work for 1 week tcpdump file ? I suspect that the > > > > memory > > > > > limitation still poses a problem, but I could do post > processing in > > > > > different machine (i.e. with 1GB Ram). It seems to me that > > > this offline > > > > > processing method should need less memory compared with real-time > > > > > processing > > > > > mode. > > > > > > > > > > > > > > > > > > > > Any feedback from people that has actually done some data > processing > > > > like > > > > > this would be appreciated :) > > > > > > > > > > > > > > > > > > > > BTW: as I saw in another post, the tcpdump file only > worked when one > > > > > specific interface is indicated with -i parameter (e.g. > > > tcpdump -i eth0 > > > > -w > > > > > dumpfile) > > > > > > > > > > > > > > > > > > > > -pfeito > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Ntop mailing list > > > > [EMAIL PROTECTED] > > > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > > > > > > _______________________________________________ > > > Ntop mailing list > > > [EMAIL PROTECTED] > > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > _______________________________________________ > > Ntop mailing list > > [EMAIL PROTECTED] > > http://listgateway.unipi.it/mailman/listinfo/ntop > > > _______________________________________________ > Ntop mailing list > [EMAIL PROTECTED] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [EMAIL PROTECTED] http://listgateway.unipi.it/mailman/listinfo/ntop
