Hi Pavol, Configuration overall looks good. Can you say how often you see that message - to understand severity of the issue? Also, what is status of the host in terms of CPU and memory utilization? CPU might be hit by the traffic rate, memory footprint can be an issue (depending on what you are monitoring) due to the aggregation method in place which can create a pretty large traffic matrix.
Couple of things i might recommend are: 1) upgrade to 0.14.3, 1.5.0rc2 or CVS head self-compiled: this will help me supporting you; 2) get a PF_RING enabled libpcap and compile pmacct against it, in case you reckon CPU utilization is the issue; 3) see if you can work around the large traffic matrix, depending on your project goals, in case you reckon memory footpring is the issue. Cheers, Paolo On Tue, Mar 11, 2014 at 10:35:00PM +0100, Pavol Gazda wrote: > Hi Paolo, > > We are running pmacct 0.14 on a Debian machine with 8GB RAM and running > Postgres as database backend. > Pmacct is used to log trafic on eth1 interface which reaches up to 1Gb. > > We are offten receiving following errors in logfile: > Mar 10 22:29:52 ERROR ( default/pgsql ): We are missing data. > Mar 10 22:29:52 If you see this message once in a while, discard it. > Otherwise some solutions follow: > Mar 10 22:29:52 - increase shared memory size, 'plugin_pipe_size'; now: > '325760000'. > Mar 10 22:29:52 - increase buffer size, 'plugin_buffer_size'; now: '325760'. > Mar 10 22:29:52 - increase system maximum socket size. > > I have tried different values of plugin_pipe_size and plugin_buffer_size > (from 81920000 - 81920 to current 325760000 - 325760) and also tried to > change sql_cache_entries from numbers close to 40 000 to 100 000 and now to > 256 000. > Could you please advise if the configuration seems ok and what we could > change to improve performance? > I would be very grateful for any help. > > pmacct.conf: > > daemonize: true > pidfile: /var/run/pmacctd.pid > logfile: /var/log/pmacct.log > aggregate: src_mac,src_host,src_port,dst_host,dst_port > pcap_filter: not net 224.0.0.0 mask 240.0.0.0 > interface: eth1 > plugins: pgsql > sql_host: localhost > sql_passwd: > sql_table: acct_%w > sql_refresh_time: 60 > sql_optimize_clauses: true > plugin_pipe_size: 325760000 > plugin_buffer_size: 325760 > sql_history: 1m > sql_history_roundoff: m > sql_dont_try_update: true > sql_cache_entries: 255989 > > Thank you in advance! > > Best regards, > > Pavol > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
