Hi Steve, You are experiencing a few connected problems, i guess. The root issue should be that the PostgreSQL database is not coping with the insert or update rate and/or with the size of the dataset.
The list of plugins you see there are, in fact, all DB writers. They are queued up, waiting for the table they want to write to will unlock. What typically happens, wrt the seg fault, is that each of these processes does take some memory; you stack many of them, memory goes away; you try to stack more, they go seg fault due to lack of system resources. You can prevent that lowering the amoung of writers allowed to stack up, by default 10, via sql_max_writers. Finally, it is normal that the active DB writer can take 100% CPU in order to dump all its data to the backend. On shortage of system resources the situation can get unstabe and you may see multiple processes in such state as they start competing with kernel, ie. swap, etc. Cheers, Paolo On Tue, Nov 08, 2016 at 03:34:57PM -0500, Stephen Clark wrote: > Hi, > > I am having a problem with nfacctd getting way behind with ver 1.5.3 > Everything is ok until I add a server that is sending a lot of netflows > then things start bogging down. I see the nfacctd plugins using 100% cpu > using top. > > Then I start getting seg faults: > Nov 8 15:28:01 netflow2 kernel: nfacctd[14296]: segfault at 0 ip > 000000000046d588 sp 00007fffe95e72c0 error 4 in > nfacctd[400000+14f000] > > 11389 ? R 8:15 nfacctd: pgsql Plugin -- DB Writer [default] > 11993 ? R 7:11 nfacctd: pgsql Plugin -- DB Writer [default] > 12229 ? R 6:14 nfacctd: pgsql Plugin -- DB Writer [default] > 12372 ? R 5:15 nfacctd: pgsql Plugin -- DB Writer [default] > 12435 ? R 4:14 nfacctd: pgsql Plugin -- DB Writer [default] > 12678 ? R 3:17 nfacctd: pgsql Plugin -- DB Writer [default] > 13187 ? R 2:17 nfacctd: pgsql Plugin -- DB Writer [default] > 13499 ? R 1:13 nfacctd: pgsql Plugin -- DB Writer [default] > 13711 ? R 0:24 nfacctd: pgsql Plugin -- DB Writer [default] > > > daemonize: true > debug: false > pidfile: /var/run/nfacctd.pid > syslog: daemon > > pre_tag_map: /etc/pmacct/my.pretag.map > > nfacctd_disable_checks: true > > nfacctd_time_new: false > > aggregate: tag, src_host, dst_host, src_port, dst_port, proto, tos > > > !plugin_pipe_size: 8192000 > !plugin_buffer_size: 8192 > plugin_pipe_size: 4096000 > plugin_buffer_size: 4096 > > plugins: pgsql > > sql_table: netflow > sql_data: typed > > sql_dont_try_update: true > sql_use_copy: true > sql_db: pmacct > sql_host: 127.0.0.1 > sql_passwd: ****** > sql_user: pmacct > > sql_refresh_time: 60 > sql_optimize_clauses: true > > sql_history: 1m > sql_history_roundoff: m > sql_recovery_logfile: /var/lib/pmacct/recovery_log > > sql_preprocess: fsrc=200000 > > sql_locking_style: row > sql_cache_entries: 299999 > > imt_buckets: 65537 > imt_mem_pools_size: 1024000 > > nfacctd_port: 2055 > > > Suggestions? > > Thanks, > Steve > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
