Hi Stathis, Since you use PF_RING, you can review an advice i gave to Joan a couple months back when he was asking how to scale up a pmacctd deployment; see specifically the replication idea i gave in the following email:
https://www.mail-archive.com/pmacct-discussion@pmacct.net/msg02447.html Speaking specifically of the classification part: gut feeling is this is a bit too much resources for only a single classifier that is looking for an HTTP hostname (i'm not necessarily implying your shared object is culprit here). It would be great if we could debug/review this together. Shall we follow-up privately on this? Cheers, Paolo On Mon, Apr 07, 2014 at 11:39:24PM +0300, Stathis Gkotsis wrote: > Hi Paolo, > Yes, I use pfring. It is both traffic rate and classification which cause the > CPU to go to 100%. If I do not use any classifiers, CPU is around 40%, then, > when I enable the classifier, CPU goes above 95%. The classifier is a shared > library which tries to match a series of bytes in the packet payload, > basically searches for a hostname in the packet payload (I am interested in > HTTP traffic). > Thanks,Stathis > > > Date: Mon, 7 Apr 2014 18:48:55 +0000 > > From: pa...@pmacct.net > > To: pmacct-discussion@pmacct.net > > Subject: Re: [pmacct-discussion] pmacct performance > > > > Hi Stathis, > > > > Two questions on your current setup: 1) are you already using pmacct > > against a PF_RING-enabled libpcap? You made reference to this in your > > email; 2) Can you determine what makes CPU go to 100%? Is it traffic > > rate or classification? Deterimining this is key to steer further > > recommendations. > > > > Cheers, > > Paolo > > > > On Sun, Apr 06, 2014 at 08:17:07PM +0300, Stathis Gkotsis wrote: > > > Hi all, > > > > > > I am using pmacctd with libpcap. My configuration is the following: > > > daemonize: falsepcap_filter: port 80 // only interested in HTTP > > > trafficplugin_pipe_size: 102400000plugin_buffer_size: 102400aggregate: > > > src_host,dst_host,src_port,dst_port,proto,classclassifiers: > > > [path_to_classifier]snaplen: 500interface: anyplugins: > > > printprint_num_protos: trueprint_cache_entries: 15485863print_output: > > > csvprint_time_roundoff: mhdprint_output_file: > > > file.%s.%Y%m%d-%H%M.txtprint_refresh_time: 300 > > > I have defined one classifier and, on the machine I am using, CPU usage > > > of the core process is close to 100%.I have read the relevant FAQ > > > question about high CPU usage and applied what it proposes. > > > The question now is how pmacct could cope with more traffic:- are there > > > any other ways to optimize pmacct itself or its configuration?- I was > > > thinking of launching multiple pmacctd instances, each instance receiving > > > a portion of the traffic. This split could be done through BPF filter. > > > How would you split the traffic? For example, you can split based on one > > > bit of the IP address... The goal would be that the separate instances > > > are balanced in terms of CPU usage.- Is pmacct compiled with all relevant > > > gcc optimizations? > > > Thanks,Stathis > > > > > _______________________________________________ > > > pmacct-discussion mailing list > > > http://www.pmacct.net/#mailinglists > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
