Hi Steve,

You are experiencing a few connected problems, i guess. The root issue
should be that the PostgreSQL database is not coping with the insert or
update rate and/or with the size of the dataset. 

The list of plugins you see there are, in fact, all DB writers. They are
queued up, waiting for the table they want to write to will unlock. What
typically happens, wrt the seg fault, is that each of these processes
does take some memory; you stack many of them, memory goes away; you try
to stack more, they go seg fault due to lack of system resources. You
can prevent that lowering the amoung of writers allowed to stack up, by
default 10, via sql_max_writers.  

Finally, it is normal that the active DB writer can take 100% CPU in order
to dump all its data to the backend. On shortage of system resources the
situation can get unstabe and you may see multiple processes in such
state as they start competing with kernel, ie. swap, etc. 

Cheers,
Paolo

On Tue, Nov 08, 2016 at 03:34:57PM -0500, Stephen Clark wrote:
> Hi,
> 
> I am having a problem with nfacctd getting way behind with ver 1.5.3
> Everything is ok until I add a server that is sending a lot of netflows
> then things start bogging down. I see the nfacctd plugins using 100% cpu 
> using top.
> 
> Then I start getting seg faults:
> Nov  8 15:28:01 netflow2 kernel: nfacctd[14296]: segfault at 0 ip
> 000000000046d588 sp 00007fffe95e72c0 error 4 in
> nfacctd[400000+14f000]
> 
> 11389 ?        R      8:15 nfacctd: pgsql Plugin -- DB Writer [default]
> 11993 ?        R      7:11 nfacctd: pgsql Plugin -- DB Writer [default]
> 12229 ?        R      6:14 nfacctd: pgsql Plugin -- DB Writer [default]
> 12372 ?        R      5:15 nfacctd: pgsql Plugin -- DB Writer [default]
> 12435 ?        R      4:14 nfacctd: pgsql Plugin -- DB Writer [default]
> 12678 ?        R      3:17 nfacctd: pgsql Plugin -- DB Writer [default]
> 13187 ?        R      2:17 nfacctd: pgsql Plugin -- DB Writer [default]
> 13499 ?        R      1:13 nfacctd: pgsql Plugin -- DB Writer [default]
> 13711 ?        R      0:24 nfacctd: pgsql Plugin -- DB Writer [default]
> 
> 
> daemonize: true
> debug: false
> pidfile: /var/run/nfacctd.pid
> syslog: daemon
> 
> pre_tag_map: /etc/pmacct/my.pretag.map
> 
> nfacctd_disable_checks: true
> 
> nfacctd_time_new: false
> 
> aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos
> 
> 
> !plugin_pipe_size: 8192000
> !plugin_buffer_size: 8192
> plugin_pipe_size: 4096000
> plugin_buffer_size: 4096
> 
> plugins: pgsql
> 
> sql_table: netflow
> sql_data: typed
> 
> sql_dont_try_update: true
> sql_use_copy: true
> sql_db: pmacct
> sql_host: 127.0.0.1
> sql_passwd: ******
> sql_user: pmacct
> 
> sql_refresh_time: 60
> sql_optimize_clauses: true
> 
> sql_history: 1m
> sql_history_roundoff: m
> sql_recovery_logfile: /var/lib/pmacct/recovery_log
> 
> sql_preprocess:  fsrc=200000
> 
> sql_locking_style: row
> sql_cache_entries: 299999
> 
> imt_buckets: 65537
> imt_mem_pools_size: 1024000
> 
> nfacctd_port: 2055
> 
> 
> Suggestions?
> 
> Thanks,
> Steve
> 
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists

_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Reply via email to