Hi Paolo, In your advice to Joan, I guess you refer to Libzero: http://www.ntop.org/products/pf_ring/libzero-for-dna/ , should not be too difficult to leverage this in pmacct... Regards,Stathis
> Date: Tue, 8 Apr 2014 16:51:00 +0000 > From: [email protected] > To: [email protected] > CC: [email protected] > Subject: Re: [pmacct-discussion] pmacct performance > > Hi Stathis, > > Since you use PF_RING, you can review an advice i gave to Joan a > couple months back when he was asking how to scale up a pmacctd > deployment; see specifically the replication idea i gave in the > following email: > > https://www.mail-archive.com/[email protected]/msg02447.html > > Speaking specifically of the classification part: gut feeling is > this is a bit too much resources for only a single classifier that > is looking for an HTTP hostname (i'm not necessarily implying your > shared object is culprit here). It would be great if we could > debug/review this together. Shall we follow-up privately on this? > > Cheers, > Paolo > > On Mon, Apr 07, 2014 at 11:39:24PM +0300, Stathis Gkotsis wrote: > > Hi Paolo, > > Yes, I use pfring. It is both traffic rate and classification which cause > > the CPU to go to 100%. If I do not use any classifiers, CPU is around 40%, > > then, when I enable the classifier, CPU goes above 95%. The classifier is a > > shared library which tries to match a series of bytes in the packet > > payload, basically searches for a hostname in the packet payload (I am > > interested in HTTP traffic). > > Thanks,Stathis > > > > > Date: Mon, 7 Apr 2014 18:48:55 +0000 > > > From: [email protected] > > > To: [email protected] > > > Subject: Re: [pmacct-discussion] pmacct performance > > > > > > Hi Stathis, > > > > > > Two questions on your current setup: 1) are you already using pmacct > > > against a PF_RING-enabled libpcap? You made reference to this in your > > > email; 2) Can you determine what makes CPU go to 100%? Is it traffic > > > rate or classification? Deterimining this is key to steer further > > > recommendations. > > > > > > Cheers, > > > Paolo > > > > > > On Sun, Apr 06, 2014 at 08:17:07PM +0300, Stathis Gkotsis wrote: > > > > Hi all, > > > > > > > > I am using pmacctd with libpcap. My configuration is the following: > > > > daemonize: falsepcap_filter: port 80 // only interested in HTTP > > > > trafficplugin_pipe_size: 102400000plugin_buffer_size: 102400aggregate: > > > > src_host,dst_host,src_port,dst_port,proto,classclassifiers: > > > > [path_to_classifier]snaplen: 500interface: anyplugins: > > > > printprint_num_protos: trueprint_cache_entries: 15485863print_output: > > > > csvprint_time_roundoff: mhdprint_output_file: > > > > file.%s.%Y%m%d-%H%M.txtprint_refresh_time: 300 > > > > I have defined one classifier and, on the machine I am using, CPU usage > > > > of the core process is close to 100%.I have read the relevant FAQ > > > > question about high CPU usage and applied what it proposes. > > > > The question now is how pmacct could cope with more traffic:- are there > > > > any other ways to optimize pmacct itself or its configuration?- I was > > > > thinking of launching multiple pmacctd instances, each instance > > > > receiving a portion of the traffic. This split could be done through > > > > BPF filter. How would you split the traffic? For example, you can > > > > split based on one bit of the IP address... The goal would be that the > > > > separate instances are balanced in terms of CPU usage.- Is pmacct > > > > compiled with all relevant gcc optimizations? > > > > Thanks,Stathis > > > > > > > _______________________________________________ > > > > pmacct-discussion mailing list > > > > http://www.pmacct.net/#mailinglists > > > > > > > > > _______________________________________________ > > > pmacct-discussion mailing list > > > http://www.pmacct.net/#mailinglists > >
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
