[pmacct-discussion] buffer overflow detected ***: nfacctd: Core Process [default] terminated
Hi. Need switch on sampling and begin get such errors: DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 269 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 268 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 269 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 268 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 269 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 268 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 269 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 268 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 269 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 268 [192.168.21.1:0]) DEBUG ( default/core ): Discarded NetFlow V9 packet (R: unknown template 269 [192.168.21.1:0]) *** buffer overflow detected ***: nfacctd: Core Process [default] terminated === Backtrace: = /lib/libc.so.6(__fortify_fail+0x40)[0xb7d74af0] /lib/libc.so.6[0xb7d72d10] nfacctd: Core Process [default](NF_counters_renormalize_handler+0xd0)[0x8062290] nfacctd: Core Process [default](exec_plugins+0x1a8)[0x805b968] nfacctd: Core Process [default](process_v9_packet+0x3a4)[0x80567a4] nfacctd: Core Process [default](main+0x1367)[0x8058957] /lib/libc.so.6(__libc_start_main+0xe6)[0xb7cadb26] nfacctd: Core Process [default][0x80551e1] === Memory map: 08048000-080ac000 r-xp fd:00 18497590 /usr/sbin/nfacctd-mysql 080ac000-080ad000 r--p 00063000 fd:00 18497590 /usr/sbin/nfacctd-mysql 080ad000-080ae000 rw-p 00064000 fd:00 18497590 /usr/sbin/nfacctd-mysql 080ae000-080da000 rw-p 080ae000 00:00 0 08ee-08f31000 rw-p 08ee 00:00 0 [heap] ab825000-ab842000 r-xp fd:00 24199173 /lib/libgcc_s.so.1 ab842000-ab843000 r--p 0001c000 fd:00 24199173 /lib/libgcc_s.so.1 ab843000-ab844000 rw-p 0001d000 fd:00 24199173 /lib/libgcc_s.so.1 ab847000-ab90b000 rw-p ab847000 00:00 0 ab90b000-ab90c000 rw-s 00:20 168834146 ab90c000-adfa8000 rw-s 00:20 168834139 adfa8000-adfa9000 rw-s 00:20 168834136 adfa9000-b0645000 rw-s 00:20 168834114 b0645000-b0646000 rw-s 00:20 168834110 b0646000-b2ce2000 rw-s 00:20 168834108 b2ce2000-b2ce3000 rw-s 00:20 168834104 b2ce3000-b537f000 rw-s 00:20 168834101 b537f000-b7a1b000 rw-s 00:20 168834095 b7a1b000-b7a1d000 rw-p b7a1b000 00:00 0 b7a1d000-b7a2f000 r-xp fd:00 10472244 /lib/libresolv-2.10.1.so b7a2f000-b7a3 r--p 00011000 fd:00 10472244 /lib/libresolv-2.10.1.so b7a3-b7a31000 rw-p 00012000 fd:00 10472244 /lib/libresolv-2.10.1.so b7a31000-b7a33000 rw-p b7a31000 00:00 0 b7a33000-b7a35000 r-xp fd:00 24199253 /lib/libkeyutils-1.2.so b7a35000-b7a36000 r--p 1000 fd:00 24199253 /lib/libkeyutils-1.2.so b7a36000-b7a37000 rw-p 2000 fd:00 24199253 /lib/libkeyutils-1.2.so b7a37000-b7a38000 rw-p b7a37000 00:00 0 b7a38000-b7a3f000 r-xp fd:00 10471725 /lib/libkrb5support.so.0.1 b7a3f000-b7a4 r--p 6000 fd:00 10471725 /lib/libkrb5support.so.0.1 b7a4-b7a41000 rw-p 7000 fd:00
Re: [pmacct-discussion] Not save data in DB when exit
26.11.2009 09:21, Paolo Lucente пишет: Hi Slava, On Wed, Nov 25, 2009 at 09:04:24PM +0200, Slava Dubrovskiy wrote: Seems when I make kill INT PID_OF_CORE_PROCESS it down, but plugins do not write to database. I see delay before off for plugins, but not see that they change command line to DB writer. And not see data for period. Can you confirm that bug? You should send a SIGINT to the plugins you want to write to the database, not only to the core process (just wondering if it's written this way in any part of the documentation). A 'killall INT pmacctd' should do it; or if you need better granularity use the 'pidfile' directive to be able to retrieve the PID for the plugins aswell. I use killall INT nfacctd and killall -s INT nfacctd and by script PIDFILE=/var/run/nfacctd.pid NAME=nfacctd LOCKFILE=/var/lock/subsys/nfacctd RETVAL=0 for P in $(ls /var/run/$NAME* 2/dev/null); do N=$(echo $P | sed s/\/var\/run\/$NAME\.pid//g) stop_daemon --displayname $NAME$N --pidfile $P --lockfile $LOCKFILE --expect-user root -2 -- nfacctd RETVAL=$? done Not work. But previos version (rc2) working good. -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Not save data in DB when exit
Hi. Seems when I make kill INT PID_OF_CORE_PROCESS it down, but plugins do not write to database. I see delay before off for plugins, but not see that they change command line to DB writer. And not see data for period. Can you confirm that bug? I use rc3 My config: daemonize: true pidfile: /var/run/nfacctd.pid syslog: daemon refresh_maps: true nfacctd_port: 8818 plugin_buffer_size: 202400 plugin_pipe_size: 2024 networks_file: /etc/pmacct/networks.list ports_file: /etc/pmacct/ports.list pre_tag_map: /etc/pmacct/pretag.map pre_tag_map_entries: 5 sql_host: 91.206.xxx.30 sql_passwd: xxx nfacctd_time_new: true sql_multi_values: 100 sql_locking_style: row sql_table_version: 4 nfacctd_renormalize: true plugins: mysql[t1], mysql[t2], mysql[t3], mysql[t4] aggregate[t1]: src_host, dst_host, src_port, dst_port, proto aggregate[t2]: tag, tag2 aggregate[t3]: src_host, dst_host, tag aggregate[t4]: tag, tag2 sql_table[t1]: acct_t1 sql_history_roundoff[t1]: h sql_history[t1]: 1h sql_refresh_time[t1]: 3600 sql_dont_try_update[t1]: true sql_recovery_logfile[t1]: /var/lib/pmacct/recovery_log_t1 sql_table[t2]: acct_t2 sql_history_roundoff[t2]: h sql_history[t2]: 1h sql_refresh_time[t2]: 3600 sql_dont_try_update[t2]: true sql_recovery_logfile[t2]: /var/lib/pmacct/recovery_log_t2 sql_table[t3]: acct_t3 sql_history_roundoff[t3]: d sql_history[t3]: 1d sql_refresh_time[t3]: 3600 sql_dont_try_update[t3]: false sql_recovery_logfile[t3]: /var/lib/pmacct/recovery_log_t3 sql_table[t4]: acct_t4 sql_history_roundoff[t4]: d sql_history[t4]: 1d sql_refresh_time[t4]: 3600 sql_dont_try_update[t4]: false sql_recovery_logfile[t4]: /var/lib/pmacct/recovery_log_t4 -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Productivity Pre-Tagging [was] Traffic count only for certain networks
24.09.2009 01:06, Paolo Lucente пишет: Hi Slava, On Wed, Sep 23, 2009 at 11:50:10PM +0300, Slava Dubrovskiy wrote: I have found other solution. With the help pre_tag_map. From networks-ua-ix.list I have made pretag.map in sort: id=1ip=192.168.21.1 filter='net 173.194.0.0/24' id=1ip=192.168.21.1 filter='net 188.163.0.0/24' id=1ip=192.168.21.1 filter='net 193.0.227.0/24' id=1ip=192.168.21.1 filter='net 193.0.228.0/24' id=1ip=192.168.21.1 filter='net 193.0.240.0/24' id=1ip=192.168.21.1 filter='net 193.0.247.0/24' id=1ip=192.168.21.1 filter='net 193.9.28.0/24' id=1ip=192.168.21.1 filter='net 193.16.45.0/24' ... Yes, that is indeed yet another viable solution :-) ---skip--- * pre_tag_map containing more than 5 entries? Dumb question Throughout our conversation about traffic accounting has noticed that periodically the daemon nfacctd hangs up. It happens when the quantity of packages strongly increases more then 50kpps (during DDoS). In log I see: Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '4333297' but received '403' collector=�^^B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '405' but received '406' collector=�^^B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '407' but received '420' collector=^H^_B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '421' but received '432' collector=^T^_B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '433' but received '446' collector=^_B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '447' but received '456' collector=,^_B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '457' but received '463' collector=3^_B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '464' but received '478' collector=B^_B:8818 agent=192.168.21.1:129 Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '479' but received '4333400' collector=X^_B:8818 agent=192.168.21.1:129 After this nfacct stop listen port and not working. Question: What occurs, when the Core Process has not time to handle all traffic? How it is possible to increase productivity Pre-Tagging? -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Traffic count only for certain networks
24.09.2009 01:06, Paolo Lucente пишет: And it put tag 1 and 0 in the same ip_src and ip_dst mysql select agent_id,ip_src,ip_dst,bytes from acct_v2 where ip_src like '91.206.226.104' and ip_dst like '91.209.165.1'; +--++--++ | agent_id | ip_src | ip_dst | bytes | +--++--++ |0 | 91.206.226.104 | 91.209.165.1 | 156040 | |1 | 91.206.226.104 | 91.209.165.1 | 19040 | +--++--++ 2 rows in set (0.01 sec) Can you explain it? A couple things come to my mind: * A different router other than 192.168.21.1 which is exporting NetFlow data to the collector. Yes! I forgot about another router. * Using NetFlow v9? If yes, any chance VLAN tags, MPLS labels or both are exported as part of the NetFlow record? In such a case you would need to refine the filter, ie. 'vlan and ...', 'mpls and ...' or 'vlan and mpls and ...'. Yes. We use NetFlow v9 from Junuper m7i and NetFlow v5 from ipcad. And we use vlan, but it's not necessary count it. What do you mean about need to refine the filter ? * Stale data? The query above doesn't show stamp_inserted and/or stamp_updated fields. They might give a clue aswell. No. I checked it. * pre_tag_map containing more than 5 entries? Dumb question but always better to ask and double check. wc -l /etc/pmacct/pretag.map 6774 /etc/pmacct/pretag.map 5 - It's with a store :) Thanks for answers -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Long filters for pretag_map
24.09.2009 20:28, Paolo Lucente пишет: Bottom line: filters in pmacct with a-la tcpdump syntax are BPF programs and obey to the libpcap rules - like tcpdump does. You should be able to gather relevant info from the local tcpdump man page; otherwise you can just follow this link http://linux.die.net/man/8/tcpdump; and look for the words vlan or mpls. Can I use long filters? For example, for tcpdump working tcpdump -n src net (91.206.226.0/23 or 91.209.165.0/24) or dst net (91.206.226.0/23 or 91.209.165.0/24) But for pretag.map this filter not work. -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Traffic count only for certain networks
23.09.2009 09:59, Paolo Lucente пишет: Hi Slava, On Tue, Sep 22, 2009 at 11:42:37PM +0300, Slava Dubrovskiy wrote: I wish to count only traffic from/to local IP. I.e. that in base were only my local IP. For this purpose I specify my local IP in networks_file and I aggregate by src_host, dst_host. I don't wish to store not local IP since then the base will be very big. Thus there are some networks (ua-list.txt) the traffic with which I need to count separately. So I make: aggregate[hourly]: src_host, dst_host aggregate[hourly_ua]: src_host, dst_host aggregate_filter[hourly_ua]: net 173.194.0.0/24 188.163.0.0/24 193.0.227.0/24 plugins: mysql[hourly], mysql[hourly_ua] sql_table[hourly]: acct_base sql_table[hourly_ua]: acct_base_ua networks_file: /etc/pmacct/networks.list But how can I separate traffic from/to UA-IX? I can use aggregate_filter for this. And it work fine. But networks list is very big :( And I can't add full list to aggregate_filter. OK: accounting for local IP addresses traffic and UA-IX networks is two different things - no traffic matrix required. Hence, i would propose you to change your config as follows: aggregate[hourly]: src_host, dst_host aggregate[hourly_ua]: src_host, dst_host plugins: mysql[hourly], mysql[hourly_ua] sql_table[hourly]: acct_base sql_table[hourly_ua]: acct_base_ua networks_file[hourly]: /etc/pmacct/networks-local.list networks_file[hourly_ua]: /etc/pmacct/networks-ua-ix.list The effect will be that the hourly_ua plugin will give you a breakdown of the traffic at the UA-IX, plus you will find a catch all entry every hour, ie. ip_src = 0, ip_dst = 0, which is the hourly sum of all the traffic between your local IP addresses and networks besides UA-IX. Many thanks for the answer. But in that case to a database will get all IP that are specified in /etc/pmacct/networks-ua-ix.list. Namely it also is to be avoided. And then to me it is not necessary [hourly] since local IP enter UA-IX But I think it the best variant that it is possible to make. I did explain advantages of usin a networks_file rather than an aggregate_filter in this scenario in a previous email (reloadable at runtime and more efficient on long list of IP). Makes sense? Yes, but to have the same mechanism for filtering it would be very good. -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Traffic count only for certain networks
10.09.2009 14:24, Slava Dubrovskiy пишет: Hi. I use nfacct in such configuration: debug: true daemonize: true pidfile: /var/run/nfacctd.pid syslog: daemon nfacctd_port: 8818 plugin_buffer_size: 10240 plugin_pipe_size: 1024000 aggregate[min]: src_host, dst_host, src_port, dst_port, proto aggregate[hourly]: src_host, dst_host --skip-- It is necessary to count the traffic with filtering on certain networks. For this purpose I need write ALL this networks to aggregate_filter. But this list of networks is big and it periodically varies. I cannot specify in aggregate_filter a file with the list of networks. How it can be made? make filter for tcpdump # cat /etc/pmacct/tcpdump-filter.txt net ( 217.175.4.0/22 || 217.73.128.0/20 || 217.76.192.0/20 || 217.77.208.0/20 ) and write aggregate_filter[hourly_ua]: -F /etc/pmacct/tcpdump-filter.txt but it's not working too :( -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] not build (was pmacct 0.12.0rc2 released !)
09.09.2009 20:00, Paolo Lucente пишет: VERSION. 0.12.0rc2 it not build. ./configure --build=i586-alt-linux --host=i586-alt-linux --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib --localstatedir=/var/lib --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --disable-dependency-tracking --without-included-gettext --with-pgsql-includes=/usr/include/pgsql --enable-64bit --enable-threads --enable-mysql --disable-pgsql --disable-sqlite3 configure: WARNING: unrecognized options: --without-included-gettext checking for a BSD-compatible install... /bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for i586-alt-linux-gcc... i586-alt-linux-gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether i586-alt-linux-gcc accepts -g... yes checking for i586-alt-linux-gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of i586-alt-linux-gcc... none checking OS... Linux checking hardware... i686 checking for i586-alt-linux-ranlib... no checking for ranlib... ranlib checking whether to enable debugging compiler options... no checking for gmake... gmake checking whether gmake sets $(MAKE)... yes checking for __progname... yes checking for extra flags needed to export symbols... --export-dynamic checking for static inline... yes checking endianess... little checking unaligned accesses... ok checking whether to disable L2
Re: [pmacct-discussion] not build (was pmacct 0.12.0rc2 released !)
09.09.2009 23:06, Paolo Lucente пишет: Hi Slava, Don't know precisely where the problem lies but i've tried the same on a Linux and a Solaris box and it works no problems: Yes. Sorry. It was my mistake. :-[ -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
