Re: [pmacct-discussion] Not save data in DB when exit
Hi Slava, On Tue, Dec 01, 2009 at 10:01:28PM +0200, Slava Dubrovskiy wrote: Yes, I remove sql_multi_values and it working. But as I understand this this give more performance. Was suspecting so correct. Is it possible use it? And it working good. Problem only when I stop it. It has to be identified what is causing it. The problem it flags is duplicates within the primary key. This is because the UPDATE query cannot be used as part of the sql_dont_try_update (which, once again gives better performances, so good it is enabled). You should get a few examples of which rows are clashing (take one duplicate error message and search for that row in the database). If stopping-starting-stopping pmacct in a very short sequence you might obvioulsy have to clean up the database for the test to be consistent. Verify whether the clash makes sense or not - given your schema and configuration. One last note: last time you posted your config, i recall you did enable 'nfacctd_time_new' which is recommended when using NetFlow in conjunction with these SQL directives. Is it still the case, right? Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Not save data in DB when exit
Hi Slava, On Mon, Nov 30, 2009 at 03:03:41PM +0200, Slava Dubrovskiy wrote: I see such errors: [ ... ] DEBUG ( t2/mysql ): 3 VALUES statements sent to the MySQL server. ERROR ( t2/mysql ): Duplicate entry '0-5-2009-11-28 02:00:00' for key 1 [ ... ] DEBUG ( t1/mysql ): 400 VALUES statements sent to the MySQL server. ERROR ( t1/mysql ): Duplicate entry '0-0.0.0.0-208.94.173.101-0-0-udp-2009-11-28 02:00:00' for key 1 Thanks for the extensive logging. You should be referring to the two errors above, right? To better pin-point where the issue is, it's good rule to remove the sql_multi_values: this allows to see precisely which SQL statement is causing the issue. From there you can check whether effectively this is something already in the database and why. The fact that you don't see anything written into the database when such errors pop up is normal due to the combination of the sql_multi_values and sql_dont_try_update directives. A multi-value query is effectively a single statement; so if one component fails it negatively affects all the others. If this doesn't help; feel free to send me further SQL output, commenting out the sql_multi_values. Perhaps this will get too extensive; in such a case, send it privately and then we can summarize over here. Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Not save data in DB when exit
26.11.2009 09:21, Paolo Lucente пишет: Hi Slava, On Wed, Nov 25, 2009 at 09:04:24PM +0200, Slava Dubrovskiy wrote: Seems when I make kill INT PID_OF_CORE_PROCESS it down, but plugins do not write to database. I see delay before off for plugins, but not see that they change command line to DB writer. And not see data for period. Can you confirm that bug? You should send a SIGINT to the plugins you want to write to the database, not only to the core process (just wondering if it's written this way in any part of the documentation). A 'killall INT pmacctd' should do it; or if you need better granularity use the 'pidfile' directive to be able to retrieve the PID for the plugins aswell. I use killall INT nfacctd and killall -s INT nfacctd and by script PIDFILE=/var/run/nfacctd.pid NAME=nfacctd LOCKFILE=/var/lock/subsys/nfacctd RETVAL=0 for P in $(ls /var/run/$NAME* 2/dev/null); do N=$(echo $P | sed s/\/var\/run\/$NAME\.pid//g) stop_daemon --displayname $NAME$N --pidfile $P --lockfile $LOCKFILE --expect-user root -2 -- nfacctd RETVAL=$? done Not work. But previos version (rc2) working good. -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Not save data in DB when exit
Hi. Seems when I make kill INT PID_OF_CORE_PROCESS it down, but plugins do not write to database. I see delay before off for plugins, but not see that they change command line to DB writer. And not see data for period. Can you confirm that bug? I use rc3 My config: daemonize: true pidfile: /var/run/nfacctd.pid syslog: daemon refresh_maps: true nfacctd_port: 8818 plugin_buffer_size: 202400 plugin_pipe_size: 2024 networks_file: /etc/pmacct/networks.list ports_file: /etc/pmacct/ports.list pre_tag_map: /etc/pmacct/pretag.map pre_tag_map_entries: 5 sql_host: 91.206.xxx.30 sql_passwd: xxx nfacctd_time_new: true sql_multi_values: 100 sql_locking_style: row sql_table_version: 4 nfacctd_renormalize: true plugins: mysql[t1], mysql[t2], mysql[t3], mysql[t4] aggregate[t1]: src_host, dst_host, src_port, dst_port, proto aggregate[t2]: tag, tag2 aggregate[t3]: src_host, dst_host, tag aggregate[t4]: tag, tag2 sql_table[t1]: acct_t1 sql_history_roundoff[t1]: h sql_history[t1]: 1h sql_refresh_time[t1]: 3600 sql_dont_try_update[t1]: true sql_recovery_logfile[t1]: /var/lib/pmacct/recovery_log_t1 sql_table[t2]: acct_t2 sql_history_roundoff[t2]: h sql_history[t2]: 1h sql_refresh_time[t2]: 3600 sql_dont_try_update[t2]: true sql_recovery_logfile[t2]: /var/lib/pmacct/recovery_log_t2 sql_table[t3]: acct_t3 sql_history_roundoff[t3]: d sql_history[t3]: 1d sql_refresh_time[t3]: 3600 sql_dont_try_update[t3]: false sql_recovery_logfile[t3]: /var/lib/pmacct/recovery_log_t3 sql_table[t4]: acct_t4 sql_history_roundoff[t4]: d sql_history[t4]: 1d sql_refresh_time[t4]: 3600 sql_dont_try_update[t4]: false sql_recovery_logfile[t4]: /var/lib/pmacct/recovery_log_t4 -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Not save data in DB when exit
Hi Slava, On Wed, Nov 25, 2009 at 09:04:24PM +0200, Slava Dubrovskiy wrote: Seems when I make kill INT PID_OF_CORE_PROCESS it down, but plugins do not write to database. I see delay before off for plugins, but not see that they change command line to DB writer. And not see data for period. Can you confirm that bug? You should send a SIGINT to the plugins you want to write to the database, not only to the core process (just wondering if it's written this way in any part of the documentation). A 'killall INT pmacctd' should do it; or if you need better granularity use the 'pidfile' directive to be able to retrieve the PID for the plugins aswell. Let me know how it goes. Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
