Hi! Adding some more details
On 06/05/13 16:01, Matthew Grant wrote: > Hi! > > Running pmacct 0.14.2 for traffic accounting here in NZ. We are a small ISP > in > the antipodes, and run all servers on NZ local time, as the country only has > one > time zone effectively. > > Found a bug in pmacct 0.14.2 with runing through a DST roll over. We were running nfacctd through April using 1 month sql_refresh_time. DST here in NZ ended 2am 7th April. nfacctd was left running over this period, and stamp_inserted for the 1st May 2013-05-01 was 23:00am!! nfacctd.conf included after patch > Looks like an > oversight. The tm_isdst flag is set to zero in the calls to mktime() in > src/util.c. It should be initialized to -1 per mktime(3) man page. This > should > work for Linux, OpenBSD, FreeBSD, NetBSD and MacOSX/Darwin. Checked use of flag on mktime() call via using python interactively and cutting and pasting between virtual machines with clock set to before and after NZ DST. > > If you want to add a config handler flag for this, it might be a good idea. Best Regards, Matthew Grant Patch is: ------- This patch sets the struct_time tm_isdst flag to -1, to tell libc to determine DST from system time zone database. This should prevent pmacct getting stamp_inserted wrong over DST roll over. This patch should work for Linux, FreeBSD, NetBSD, OpenBSD and MacOSX/Darwin. Matthew Grant <[email protected]> Mon, 06 May 2013 14:46:38 +1200 Index: pmacct-0.14.2/src/util.c =================================================================== --- pmacct-0.14.2.orig/src/util.c 2013-05-06 14:50:36.811097741 +1200 +++ pmacct-0.14.2/src/util.c 2013-05-06 14:53:46.281424257 +1200 @@ -330,6 +330,8 @@ } } + /* Tell libc to sort DST rollover when calculating over start/end of DST */ + rounded->tm_isdst = -1; t = mktime(rounded); return t; } @@ -349,6 +351,8 @@ howmany--; } + /* Tell libc to sort DST rollover when calculating over start/end of DST */ + tmt->tm_isdst = -1; final = mktime(tmt); return (final-base); ---- ! nfacctd configuration ! ! ! daemonize: false syslog: daemon nfacctd_port: 9996 pidfile: /var/run/pmacct/nfacctd.pid ! Adjust nfacctd core process receiver queue plugin_pipe_size: 40960000 plugin_buffer_size: 40960 plugin_pipe_backlog: 25 pre_tag_map: /var/lib/ipac/pretag.map refresh_maps: true ! Need to use AS file to deconvolute pretag.map - Sheesh, pmacct can have a config brain fuck networks_file: /var/lib/ipac/pretag-networks.conf nfacctd_as_new: file ! ! storage methods plugins: pgsql[1h_ip_in], pgsql[1h_ip_out], pgsql[1d_ip_in], pgsql[1d_ip_out], pgsql[1month_ip_in], pgsql[1month_ip_out], pgsql[1h_ip_in_country], pgsql[1h_ip_out_country], pgsql[1d_ip_in_country], pgsql[1d_ip_out_country], pgsql[1month_ip_in_country],pgsql[1month_ip_out_country], pgsql[1h_ipip], pgsql[1h_detail], pgsql[1h_type], pgsql[5m_ip_in_country], pgsql[5m_ip_out_country], pgsql[5m_ip_in], pgsql[5m_ip_out] ! DB details ! use uinix socket as it is most effecient. sql_host: /var/run/postgresql sql_passwd: 1RAeLODtp01Z508K ! BGP setup !bgp_daemon: true !bgp_daemon_ip: 111.65.238.13 !bgp_daemon_max_peers: 25 !bgp_daemon_msglog: true !nfacctd_as_new: false !nfacctd_as_new: bgp ! Default table values sql_locking_style: table sql_optimize_clauses: true aggregate: src_host, src_port, dst_host, dst_port, proto, tag nfacctd_time_new: true nfacctd_renormalize: true ! Only need this for wierd netflow sources sampling_map: /etc/ipac/sampling-rate.map ! Could be good for deconvolving IPv6 if I knew how it worked. !networks_mask: 64 geoip_ipv4_file: /usr/share/GeoIP/GeoIP.dat geoip_ipv6_file: /usr/share/GeoIP/GeoIPv6.dat sql_max_writers: 25 sql_delimiter: | ! Prevent running out of buckets, and dropped traffic accounting entries sql_cache_entries: 193939 ! BGP hourly table !sql_table[1h_bgp]: acct_1h_bgp !aggregate[1h_bgp]: src_as, dst_as, peer_src_as, peer_dst_as, peer_src_ip, peer_dst_ip, as_path, local_pref, med, tag, tag2 !sql_history[1h_bgp]: 1h !sql_history_roundoff[1h_bgp]: h !sql_refresh_time[1h_bgp]: 300 !sql_dont_try_update[1h_bgp]: true !sql_use_copy[1h_bgp]: true ! Per host summary, in, hourly sql_table[1h_ip_in]: acct_1h_ip_in aggregate[1h_ip_in]: dst_host, tag networks_file[1h_ip_in]: /var/lib/ipac/prefix-networks.conf sql_history[1h_ip_in]: 1h sql_history_roundoff[1h_ip_in]: h sql_refresh_time[1h_ip_in]: 300 pre_tag2_filter[1h_ip_in]: 1 ! Per host summary, out, hourly sql_table[1h_ip_out]: acct_1h_ip_out aggregate[1h_ip_out]: src_host, tag networks_file[1h_ip_out]: /var/lib/ipac/prefix-networks.conf sql_history[1h_ip_out]: 1h sql_history_roundoff[1h_ip_out]: h sql_refresh_time[1h_ip_out]: 300 pre_tag2_filter[1h_ip_out]: 2 ! Per host summary, in sql_table[1d_ip_in]: acct_1d_ip_in aggregate[1d_ip_in]: dst_host, tag networks_file[1d_ip_in]: /var/lib/ipac/prefix-networks.conf sql_history[1d_ip_in]: 1d sql_history_roundoff[1d_ip_in]: d sql_refresh_time[1d_ip_in]: 300 pre_tag2_filter[1d_ip_in]: 1 ! Per host summary, out sql_table[1d_ip_out]: acct_1d_ip_out aggregate[1d_ip_out]: src_host, tag networks_file[1d_ip_out]: /var/lib/ipac/prefix-networks.conf sql_history[1d_ip_out]: 1d sql_history_roundoff[1d_ip_out]: d sql_refresh_time[1d_ip_out]: 300 pre_tag2_filter[1d_ip_out]: 2 ! Per host summary, in, monthly sql_table[1month_ip_in]: acct_1month_ip_in aggregate[1month_ip_in]: dst_host, tag networks_file[1month_ip_in]: /var/lib/ipac/prefix-networks.conf sql_history[1month_ip_in]: 1M sql_history_roundoff[1month_ip_in]: M sql_refresh_time[1month_ip_in]: 300 pre_tag2_filter[1month_ip_in]: 1 ! Per host summary, out, monthly sql_table[1month_ip_out]: acct_1month_ip_out aggregate[1month_ip_out]: src_host, tag networks_file[1month_ip_out]: /var/lib/ipac/prefix-networks.conf sql_history[1month_ip_out]: 1M sql_history_roundoff[1month_ip_out]: M sql_refresh_time[1month_ip_out]: 300 pre_tag2_filter[1month_ip_out]: 2 ! Per host summary, in, percentile 5 minutes sql_table[5m_ip_in_country]: acct_5m_ip_in_country aggregate[5m_ip_in_country]: dst_host, src_host_country, tag networks_file[5m_ip_in_country]: /var/lib/ipac/prefix-networks.conf sql_history[5m_ip_in_country]: 5m sql_history_roundoff[5m_ip_in_country]: m sql_refresh_time[5m_ip_in_country]: 300 sql_dont_try_update[5m_ip_in_country]: true sql_use_copy[5m_ip_in_country]: true pre_tag2_filter[5m_ip_in_country]: 1 ! Per host summary, out, percentile 5 minutes sql_table[5m_ip_out_country]: acct_5m_ip_out_country aggregate[5m_ip_out_country]: src_host, dst_host_country, tag networks_file[5m_ip_out_country]: /var/lib/ipac/prefix-networks.conf sql_history[5m_ip_out_country]: 5m sql_history_roundoff[5m_ip_out_country]: m sql_refresh_time[5m_ip_out_country]: 300 sql_dont_try_update[5m_ip_out_country]: true sql_use_copy[5m_ip_out_country]: true pre_tag2_filter[5m_ip_out_country]: 2 ! Per host summary, in, domestic hourly sql_table[1h_ip_in_country]: acct_1h_ip_in_country aggregate[1h_ip_in_country]: dst_host, src_host_country, tag networks_file[1h_ip_in_country]: /var/lib/ipac/prefix-networks.conf sql_history[1h_ip_in_country]: 1h sql_history_roundoff[1h_ip_in_country]: h sql_refresh_time[1h_ip_in_country]: 300 pre_tag2_filter[1h_ip_in_country]: 1 ! Per host summary, out, domestic hourly sql_table[1h_ip_out_country]: acct_1h_ip_out_country aggregate[1h_ip_out_country]: src_host, dst_host_country, tag networks_file[1h_ip_out_country]: /var/lib/ipac/prefix-networks.conf sql_history[1h_ip_out_country]: 1h sql_history_roundoff[1h_ip_out_country]: h sql_refresh_time[1h_ip_out_country]: 300 pre_tag2_filter[1h_ip_out_country]: 2 ! Per host summary, in, domestic daily sql_table[1d_ip_in_country]: acct_1d_ip_in_country aggregate[1d_ip_in_country]: dst_host, src_host_country, tag networks_file[1d_ip_in_country]: /var/lib/ipac/prefix-networks.conf sql_history[1d_ip_in_country]: 1d sql_history_roundoff[1d_ip_in_country]: d sql_refresh_time[1d_ip_in_country]: 300 pre_tag2_filter[1d_ip_in_country]: 1 ! Per host summary, out, domestic daily sql_table[1d_ip_out_country]: acct_1d_ip_out_country aggregate[1d_ip_out_country]: src_host, dst_host_country, tag networks_file[1d_ip_out_country]: /var/lib/ipac/prefix-networks.conf sql_history[1d_ip_out_country]: 1d sql_history_roundoff[1d_ip_out_country]: d sql_refresh_time[1d_ip_out_country]: 300 pre_tag2_filter[1d_ip_out_country]: 2 ! Per host summary, in, domestic monthly sql_table[1month_ip_in_country]: acct_1month_ip_in_country aggregate[1month_ip_in_country]: dst_host, src_host_country, tag networks_file[1month_ip_in_country]: /var/lib/ipac/prefix-networks.conf sql_history[1month_ip_in_country]: 1M sql_history_roundoff[1month_ip_in_country]: M sql_refresh_time[1month_ip_in_country]: 300 pre_tag2_filter[1month_ip_in_country]: 1 ! Per host summary, out, domestic monthly sql_table[1month_ip_out_country]: acct_1month_ip_out_country aggregate[1month_ip_out_country]: src_host, dst_host_country, tag networks_file[1month_ip_out_country]: /var/lib/ipac/prefix-networks.conf sql_history[1month_ip_out_country]: 1M sql_history_roundoff[1month_ip_out_country]: M sql_refresh_time[1month_ip_out_country]: 300 pre_tag2_filter[1month_ip_out_country]: 2 ! Hourly IP to IP file sql_table[1h_ipip]: acct_1h_ipip aggregate[1h_ipip]: src_host, dst_host, tag, tag2 ports_file[1h_ipip]: /var/lib/ipac/ports.conf sql_history[1h_ipip]: 1h sql_history_roundoff[1h_ipip]: h sql_refresh_time[1h_ipip]: 3600 networks_file[1h_ipip]: /etc/ipac/default-route-networks.conf sql_dont_try_update[1h_ipip]: true sql_use_copy[1h_ipip]: true pre_tag2_filter[1h_ipip]: 1,2,5 ! Hourly detail table sql_table[1h_detail]: acct_1h_detail aggregate[1h_detail]: src_host, src_port, dst_host, dst_port, proto, tag, tag2 ports_file[1h_detail]: /var/lib/ipac/ports.conf sql_history[1h_detail]: 1h sql_history_roundoff[1h_detail]: h sql_refresh_time[1h_detail]: 3600 networks_file[1h_detail]: /etc/ipac/default-route-networks.conf sql_dont_try_update[1h_detail]: true sql_use_copy[1h_detail]: true sql_table[1h_type]: acct_1h_type aggregate[1h_type]: src_port, dst_port, proto, flows, tag2 sql_history[1h_type]: 1h sql_history_roundoff[1h_type]: h sql_refresh_time[1h_type]: 3600 networks_file[1h_type]: /etc/ipac/default-route-networks.conf sql_dont_try_update[1h_type]: true sql_use_copy[1h_type]: true pre_tag2_filter[1h_type]: 1,2,5 ! Per host summary, in, percentile 5 minutes sql_table[5m_ip_in_country]: acct_5m_ip_in_country aggregate[5m_ip_in_country]: dst_host, src_host_country, tag networks_file[5m_ip_in_country]: /var/lib/ipac/prefix-networks.conf sql_history[5m_ip_in_country]: 5m sql_history_roundoff[5m_ip_in_country]: m sql_refresh_time[5m_ip_in_country]: 300 sql_dont_try_update[5m_ip_in_country]: true sql_use_copy[5m_ip_in_country]: true pre_tag2_filter[5m_ip_in_country]: 1 ! Per host summary, out, percentile 5 minutes sql_table[5m_ip_out_country]: acct_5m_ip_out_country aggregate[5m_ip_out_country]: src_host, dst_host_country, tag networks_file[5m_ip_out_country]: /var/lib/ipac/prefix-networks.conf sql_history[5m_ip_out_country]: 5m sql_history_roundoff[5m_ip_out_country]: m sql_refresh_time[5m_ip_out_country]: 300 sql_dont_try_update[5m_ip_out_country]: true sql_use_copy[5m_ip_out_country]: true pre_tag2_filter[5m_ip_out_country]: 2 ! Per host summary, in, percentile 5 minutes sql_table[5m_ip_in]: acct_5m_ip_in aggregate[5m_ip_in]: dst_host, tag networks_file[5m_ip_in]: /var/lib/ipac/prefix-networks.conf sql_history[5m_ip_in]: 5m sql_history_roundoff[5m_ip_in]: m sql_refresh_time[5m_ip_in]: 300 sql_dont_try_update[5m_ip_in]: true sql_use_copy[5m_ip_in]: true pre_tag2_filter[5m_ip_in]: 1 ! Per host summary, out, percentile 5 minutes sql_table[5m_ip_out]: acct_5m_ip_out aggregate[5m_ip_out]: src_host, tag networks_file[5m_ip_out]: /var/lib/ipac/prefix-networks.conf sql_history[5m_ip_out]: 5m sql_history_roundoff[5m_ip_out]: m sql_refresh_time[5m_ip_out]: 300 sql_dont_try_update[5m_ip_out]: true sql_use_copy[5m_ip_out]: true pre_tag2_filter[5m_ip_out]: 2 _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
