Re: [pmacct-discussion] Next-hop not populated when using networks file
Ok, I think I got it now (still not workin though), there where several
wrong assumptions from my part:
- Next hop is only (logicaly) stored for outgoing packets
- I am using nfsen (ncapd) to capture the flows, by default, nfcapd
captures netflow v9 but only extensions 1 (input/output interface SNMP
numbers) and 2 (src/dst AS numbers), the nex-hop ip address is extension 4.
So I had to reconfigure nfsen so it added -T +4 to the nfcapd daemon
- A very nice way to debug the flow data is by using tshark (even on non
standard ports):
tshark -i eth1 host 192.168.1.22 -d udp.port==2591,cflow -s0 -V
Thanks for all your help,
Joan
2014-04-07 20:56 GMT+02:00 Paolo Lucente pa...@pmacct.net:
Hi Joan,
I've just tried to reproduce the issue with latest CVS with
no luck, ie. BGP next-hop information is inserted just fine.
If you make a pcap capture of the NetFlow traffic produced
by nfprobe (or are able to debug NetFlow v9 templates in the
collector tool) do you reckon the BGP next-hop field is part
of the template (and hence left as 0.0.0.0)?
Cheers,
Paolo
On Mon, Apr 07, 2014 at 04:37:29PM +0200, Joan wrote:
Just tried it, it seems that pmacct isn't yet adding th nexthop
information, this is my current config, I added the
peer_src_ip,peer_dst_ip
primitives and the nfacctd_net: file, maybe I'm missing something
! pmacctd configuration
!
!
!
daemonize: true
pidfile: /var/run/pmacctd.pid
syslog: daemon
!
! interested in in and outbound traffic
!aggregate: src_host,dst_host,dst_as,src_as,src_port,dst_port,proto,tos
aggregate:
src_host,dst_host,dst_as,src_as,src_port,dst_port,proto,tos,peer_src_ip,peer_dst_ip
! on this network
!pcap_filter: net 0.0.0.0/0
! on this interface
interface: eth0
!
plugins: nfprobe
networks_file: /etc/pmacct/networks.lst
refresh_maps: true
nfprobe_receiver: 192.168.1.123:2591
nfprobe_version: 9
pmacctd_as: file
!added after last email
nfacctd_net: file
!plugin_pipe_size: 2048000
!plugin_buffer_size: 2048
plugin_pipe_size: 4096000
plugin_buffer_size: 4096
debug : false
Sample file:
123.123.123.123,17766,223.255.235.0/24
123.123.123.123,56000,223.255.236.0/24
123.123.123.123,56000,223.255.237.0/24
123.123.123.123,56000,223.255.238.0/24
123.123.123.123,56000,223.255.239.0/24
123.123.123.123,55649,223.255.240.0/22
123.123.123.123,55649,223.255.240.0/24
123.123.123.123,55649,223.255.241.0/24
123.123.123.123,55649,223.255.242.0/24
123.123.123.123,55649,223.255.243.0/24
123.123.123.123,45954,223.255.244.0/24
123.123.123.123,45954,223.255.245.0/24
123.123.123.123,45954,223.255.246.0/24
123.123.123.123,45954,223.255.247.0/24
123.123.123.123,55415,223.255.254.0/24
2014-04-07 16:16 GMT+02:00 Joan aseq...@gmail.com:
The date I've in the checkout folder is Feb, 17th, and it's probably
from
those days (also it's trunk code), I'll update to current head and
test it
again.
2014-04-05 4:22 GMT+02:00 Paolo Lucente pa...@pmacct.net:
Hi Joan,
Can you confirm you do not run a CVS build past Feb, 5th
and you want the BGP next-hop taken from a networks_file
in conjunction with the nfprobe plugin? If yes, you should
be sorted if downloading latest CVS:
https://www.mail-archive.com/pmacct-commits@pmacct.net/msg00981.html
For the BGP next-hop to be taken from a networks_file you
should also configure nfacctd_net to 'file': as you might
see from docs that's the one influencing 'peer_dst_ip' (or
BGP next-hop). Let me know if this is of help.
Cheers,
Paolo
On Fri, Apr 04, 2014 at 11:39:28AM +0200, Joan wrote:
I am using a networks_file such as this, being the next hop
123.123.123.123, I do have other bgp providers for other routes.
123.123.123.123,17766,223.255.235.0/24
123.123.123.123,56000,223.255.236.0/24
123.123.123.123,56000,223.255.237.0/24
123.123.123.123,56000,223.255.238.0/24
123.123.123.123,56000,223.255.239.0/24
123.123.123.123,55649,223.255.240.0/22
123.123.123.123,55649,223.255.240.0/24
123.123.123.123,55649,223.255.241.0/24
123.123.123.123,55649,223.255.242.0/24
123.123.123.123,55649,223.255.243.0/24
123.123.123.123,45954,223.255.244.0/24
123.123.123.123,45954,223.255.245.0/24
123.123.123.123,45954,223.255.246.0/24
123.123.123.123,45954,223.255.247.0/24
123.123.123.123,55415,223.255.254.0/24
The issue I am having is that altough the AS numbers are properly
populated, the BGPNextHop field is always 0.0.0.0
I am using this aggregate list:
aggregate:
src_host,dst_host,dst_as,src_as,src_port,dst_port,proto,tos,peer_src_ip,peer_dst_ip
From the config keys (http://wiki.pmacct.net/OfficialConfigKeys) i
read:
when 'true' ('file' being an alias of 'true') it instructs
Re: [pmacct-discussion] Network file not properly load
Ok, then I will try to adapt the script I used to generate this file, because there are lots of routes saved from the bgp summary. https://github.com/paololucente/pmacct-contrib/blob/master/st1/quagga_gen_as_network.pl 2014-02-13 16:39 GMT+01:00 Adam Bogdan nelr...@gmail.com: Hi Joan, The problem is with this 2 lines: 123.123.123.123,55649,223.255.240.0/22 123.123.123.123,55649,223.255.240.0/24 Just delete the line with /24 and check then - I had similar problem with overlapping prefixes. Regards Adam 2014-02-13 15:36 GMT+01:00 Joan aseq...@gmail.com: While loading the attached network file, I get this strange errors on the logs (when debug is enabled), it seems that the networks are not properly imported (it seems related to the nested networks) but I couldn't simplify any more the test case. The problem is that when there are those errors the srcas and dstas never get populated on the flows. Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 17766 net: 223.255.235.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 45954 net: 223.255.244.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 45954 net: 223.255.245.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 45954 net: 223.255.246.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 45954 net: 223.255.247.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 55415 net: 223.255.254.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: peer asn: 0 asn: 0 net: 0.0.0.0 mask: 0 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] contains a default route Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: peer asn: 0 asn: 0 net: 0.0.0.0 mask: 0 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] contains a default route Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: peer asn: 0 asn: 0 net: 0.0.0.0 mask: 0 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] contains a default route Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: peer asn: 0 asn: 0 net: 0.0.0.0 mask: 0 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] contains a default route Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: peer asn: 0 asn: 0 net: 0.0.0.0 mask: 0 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] contains a default route Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 55649 net: 223.255.240.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 55649 net: 223.255.241.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 55649 net: 223.255.242.0 mask: 24 Feb 13 15:31:07 collector pmacctd[29186]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 123.123.123.123 peer asn: 0 asn: 55649 net: 223.255.243.0 mask: 24 ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Strange results on nfdump when using networks_file
Hi Paolo, this is the exact steps I am doing to compile all the setup, one important thing is that the version I get on the cvs checkout is older than yours. Mine is 1.24, from 2014-01-09 I documented all the steps in this gist: https://gist.github.com/aseques/8912310 Text in the cvs head is this one: RCS file: /home/repo-0.14/pmacct/AUTHORS,v Working file: AUTHORS head: 1.24 branch: locks: strict access list: symbolic names: start: 1.1.1.1 plucente: 1.1.1 keyword substitution: kv total revisions: 25;selected revisions: 25 description: revision 1.24 date: 2014-01-09 19:54:54 +0100; author: paolo; state: Exp; lines: +3 -2; commitid: lbgI3khZiZPsrtkx; * fix, MySQL plugin: added linking of pmacct code against -lstdc++ and -lrt if MySQL plugin is enabled, pre-requisite for MySQL 5.6. Many thanks to Stefano Birmani for reporting the issue. 2014-02-07 12:28 GMT+01:00 Paolo Lucente pa...@pmacct.net: Hi Joan, Just fetched myself for a test from the CVS and build is 20140205-00 (ie. pmacctd -V). Do i get correctly the issue is: you are able to compile pmacct 1.5.0rc2 against PF_RING 5.6.2 but not latest pmacct code from CVS against the exact same PF_RING library? Cheers, Paolo On Fri, Feb 07, 2014 at 11:59:02AM +0100, Joan wrote: Hi Paolo, the code from the cvs (module pmacct) doesn't compile when linking with pfring enabled libpcap, it does without problem when there is only the system libpcap, see the log attached. It was when using PF_RING-5.6.2 Still the las commit to the cvs is from 2014-01-09, earlier than my mail, is this the proper module to checkout from? Regards, Joan 2014-02-05 17:42 GMT+01:00 Paolo Lucente pa...@pmacct.net: Hi Joan, I verified the issue you describe and fixed in the CVS. Can you give it a try and see if that works for you? Cheers, Paolo On Wed, Feb 05, 2014 at 11:50:55AM +0100, Joan wrote: I am trying to setup again a system to export flows with as number by using the networks_file, since creating a full networks_file with the script at ( https://github.com/paololucente/pmacct-contrib/tree/master/st1) failed leaving all the AS fields as 0, I simplified the file to a minimal case (only google's 8.8.8.x and 8.8.4.x) ! generated by quagga_gen_as_network.pl at 20140205-11:25.51 193.149.55.94,15169,8.8.4.0/24 193.149.55.94,15169,8.8.8.0/24 Now I'm getting the srcas and dstas setted for all the traffic as if it was originated and destinated to google. I'm using the current 1.5.0rc2. Feb 5 11:37:43 flower pmacctd[9562]: INFO ( default/core ): Start logging ... Feb 5 11:37:43 flower pmacctd[9562]: INFO ( default/nfprobe ): plugin_pipe_size=4096000 bytes plugin_buffer_size=4096 bytes Feb 5 11:37:43 flower pmacctd[9562]: INFO ( default/nfprobe ): ctrl channel: obtained=163840 bytes target=4000 bytes Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 193.150.1.123 peer asn: 0 asn: 15169 net: 8.8.4.0 mask: 24 Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 software, Copyright 2002 Damien M iller d...@mindrot.org All rights reserved. Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 193.150.1.123 peer asn: 0 asn: 15169 net: 8.8.8.0 mask: 24 Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): TCP timeout: 3600s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): TCP post-RST timeout: 120s Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): IPv4 Networks Cache successfully created: 1 entries. Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): TCP post-FIN timeout: 300s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): UDP timeout: 300s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): ICMP timeout: 300s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): General timeout: 3600s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): Maximum lifetime: 604800s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): Expiry interval: 60s Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv6] nh: 193.150.1.123 peer_asn: 0 asn: 15169 net: :: mask: 0 Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv6] contains a default route Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): IPv6 Networks Cache successfully created: 32771 entries. Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): Exporting flows to [192.168.1.123]:2591
[pmacct-discussion] Strange results on nfdump when using networks_file
I am trying to setup again a system to export flows with as number by using the networks_file, since creating a full networks_file with the script at ( https://github.com/paololucente/pmacct-contrib/tree/master/st1) failed leaving all the AS fields as 0, I simplified the file to a minimal case (only google's 8.8.8.x and 8.8.4.x) ! generated by quagga_gen_as_network.pl at 20140205-11:25.51 193.149.55.94,15169,8.8.4.0/24 193.149.55.94,15169,8.8.8.0/24 Now I'm getting the srcas and dstas setted for all the traffic as if it was originated and destinated to google. I'm using the current 1.5.0rc2. Feb 5 11:37:43 flower pmacctd[9562]: INFO ( default/core ): Start logging ... Feb 5 11:37:43 flower pmacctd[9562]: INFO ( default/nfprobe ): plugin_pipe_size=4096000 bytes plugin_buffer_size=4096 bytes Feb 5 11:37:43 flower pmacctd[9562]: INFO ( default/nfprobe ): ctrl channel: obtained=163840 bytes target=4000 bytes Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 193.150.1.123 peer asn: 0 asn: 15169 net: 8.8.4.0 mask: 24 Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 software, Copyright 2002 Damien M iller d...@mindrot.org All rights reserved. Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 193.150.1.123 peer asn: 0 asn: 15169 net: 8.8.8.0 mask: 24 Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): TCP timeout: 3600s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): TCP post-RST timeout: 120s Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): IPv4 Networks Cache successfully created: 1 entries. Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): TCP post-FIN timeout: 300s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): UDP timeout: 300s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): ICMP timeout: 300s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): General timeout: 3600s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): Maximum lifetime: 604800s Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): Expiry interval: 60s Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv6] nh: 193.150.1.123 peer_asn: 0 asn: 15169 net: :: mask: 0 Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv6] contains a default route Feb 5 11:37:43 flower pmacctd[9562]: DEBUG ( /etc/pmacct/networks.lst ): IPv6 Networks Cache successfully created: 32771 entries. Feb 5 11:37:43 flower pmacctd[9563]: INFO ( default/nfprobe ): Exporting flows to [192.168.1.123]:2591 Feb 5 11:37:43 flower pmacctd[9563]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 193.150.1.123 peer asn: 0 asn: 15169 net: 8.8.4.0 mask: 24 Feb 5 11:37:43 flower pmacctd[9563]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv4] nh: 193.150.1.123 peer asn: 0 asn: 15169 net: 8.8.8.0 mask: 24 Feb 5 11:37:43 flower pmacctd[9563]: DEBUG ( /etc/pmacct/networks.lst ): IPv4 Networks Cache successfully created: 1 entries. Feb 5 11:37:43 flower pmacctd[9563]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv6] nh: 193.150.1.123 peer_asn: 0 asn: 15169 net: :: mask: 0 Feb 5 11:37:43 flower pmacctd[9563]: DEBUG ( /etc/pmacct/networks.lst ): [networks table IPv6] contains a default route Feb 5 11:37:43 flower pmacctd[9563]: DEBUG ( /etc/pmacct/networks.lst ): IPv6 Networks Cache successfully created: 32771 entries. Feb 5 11:37:43 flower pmacctd[9562]: OK ( default/core ): link type is: 1 Dst IP Addr FlowsBytes Packets Src AS Dst AS 209.23.235.22 1 921 15169 15169 88.26.252.71 1 3855 15169 15169 166.78.151.214 1 871 15169 15169 88.26.252.71 1 4185 15169 15169 162.242.162.82 1 811 15169 15169 69.28.95.170 1 801 15169 15169 69.28.95.154 1 781 15169 15169 218.189.3.34 1 761 15169 15169 64.132.253.13 1 741 15169 15169 88.26.252.71 1 4185 15169 15169 195.55.157.82 1 1561 15169 15169 205.251.194.67 1 861 15169 15169 88.26.252.71 1 4185 15169 15169 178.79.150.32 1 921 15169 15169 176.58.111.122 1 921 15169 15169 209.59.139.12 1 731 15169 15169 178.79.150.32 1 1101 15169 15169 54.248.92.63 1 761 15169 15169 networks.lst Description: Binary data ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Pmacct not adding the as number
Hello again, with the changes you proposed it seems to work fine, the only missing AS i see now are from our own bgp system, I imagine that's because they aren't saved in the output of sh ip bgp, so it can be easily fixed. 2013/7/9 Joan aseq...@gmail.com I have done the change, I don't have yet any significant amount of flows to analyze, so I'll be back later when I have more information. Thanks a lot for your help, Joan 2013/7/9 Paolo Lucente pa...@pmacct.net Hi Joan, Please add 'pmacctd_as: file' to your config. Actually, in absence of any config directive at this propo, this should be the default setting (if, of course, a networks_file is loaded and we speak pmacctd daemon). Will reproduce your config in lab and see why that would not be happening. Cheers, Paolo On Tue, Jul 09, 2013 at 02:56:30PM +0200, Joan wrote: Hi again, I am reopening this thread again because after upgrading to current 0.14.3 version (which fixes all my crashes) the srcas/dstas data still isn't populated. This is my current config: daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon aggregate: src_host,dst_host,dst_as,src_as interface: br0 plugins: nfprobe networks_file: /etc/pmacct/networks.lst nfprobe_receiver: 192.168.1.5:2591 nfprobe_version: 9 debug : true See the attached document for the dump of the flows that I am doing in the flow collector. And an sample entry in the networks.lst file for one of the matches in the file: 29073,80.82.64.0/24 2013/7/5 Paolo Lucente pa...@pmacct.net xHi Joan, Thanks for explaining the background, it makes sense. To get ASNs info populated you should add src_as and dst_as primitives to your aggregate directive. Same as any further info you wish to see populated. Let me know how that goes. I see you dropped a separate email about a crash, along with a backtrace, thanks for that. I will look into it, ie. maybe you already hinted the above yourself and got to the next stage, and get back to you. Cheers, Paolo On Fri, Jul 05, 2013 at 02:35:15PM +0200, Joan wrote: Hello, I am trying to get pmacct workting to replace softflowd because we'd like to have the as numbers for the networks populated. To accomplish this I am using the script to generate the networks_file from quagga (I had a couple of issues but it's ok now) This is my pmactd.conf config: /etc/pmacct/pmactd.conf daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon aggregate: src_host,dst_host pcap_filter: net 0.0.0.0/0 interface: br0 plugins: nfprobe nfprobe_version: 9 networks_file: /etc/pmacct/networks.lst nfprobe_receiver: 192.168.1.5:2591 http://192.168.1.8:2591 nfprobe_version: 9 And in the flow collector I am checking for the as numbers with nfdump, but the output of srcas/dstas is always 0 nfdump -A srcas -N -M /var/lib/netflow/profiles-data/live/ -o fmt:%sa %fl %byt %pkt %sas %das -R nfcapd.201307051420:nfcapd.201307051425 Did I miss something in the pmacctd config? I don't see anything relevant in the logs. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists nfdump -M /var/lib/netflow/profiles-data/live/ -R nfcapd.201307091410:nfcapd.201307091440 -A srcas,dstas,srcip,dstip Date flow start Duration Src AS Dst AS Src IP Addr Dst IP Addr PacketsBytes bpsBpp Flows 2013-07-09 13:30:18.67914.592 0 0218.94.15.226 123.123.123.1233 120 65 40 1 2013-07-09 14:07:54.345 3.094 0 0 92.81.226.61 123.123.123.1232 96 248 48 1 2013-07-09 14:32:49.080 0.000 0 0 188.165.95.171 123.123.123.1241 440 44 1 2013-07-09 09:20:01.379 18867.828 0 023.123.123.25 224.0.0.6 1473 110892 47 75 1 2013-07-09 13:21:32.957 0.000 0 0 85.237.35.52 123.123.123.1231 600 60 1 2013-07-09 14:26:16.360 0.000 0 0 80.82.64.231 123.123.123.1241 290 29 1 2013-07-09 13:47:01.881 0.000 0 0 186.202.186.28 123.123.123.1241 520 52 1 2013-07-09 09:19:59.525 18878.256 0 0 123.123.123.25 224.0.0.5 1889 151120 64 80 1 2013-07-09 13:28:24.305 0.000 0 0 61.147.103.117 123.123.123.1231 400 40 1 ___ pmacct-discussion mailing list http
Re: [pmacct-discussion] Crash in pmacct
Hello, most of the patches from debian are issues related to packaging for other architectures or issues with the location of docs, and so on, nothing really relevant to code as I saw. So it's mostly a pristine 0.14.0 which seems pretty old after reading about newer versions. In any case 0.14.3 compiles just fine, and doesn't crash anymore, if someone wants the .deb just ask :) I'm still having issues populating the srcas, dstas, but that's for another thread. 2013/7/8 Karl O. Pinc k...@meme.com On 07/08/2013 05:30:36 AM, Joan wrote: BTW, just found in the changelog for 0.14.1 this: ! fix, net_aggr.c: defining a networks_file configuration directive in conjunction with --enable-ipv6 was causing a SEGVs. This is now solved. That could be the cause for my issue (unless debian backported the fixes) See /usr/share/doc/pmacct*/changelog.Debian* to check for backports. Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Pmacct not adding the as number
Hi again, I am reopening this thread again because after upgrading to current 0.14.3 version (which fixes all my crashes) the srcas/dstas data still isn't populated. This is my current config: daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon aggregate: src_host,dst_host,dst_as,src_as interface: br0 plugins: nfprobe networks_file: /etc/pmacct/networks.lst nfprobe_receiver: 192.168.1.5:2591 nfprobe_version: 9 debug : true See the attached document for the dump of the flows that I am doing in the flow collector. And an sample entry in the networks.lst file for one of the matches in the file: 29073,80.82.64.0/24 2013/7/5 Paolo Lucente pa...@pmacct.net xHi Joan, Thanks for explaining the background, it makes sense. To get ASNs info populated you should add src_as and dst_as primitives to your aggregate directive. Same as any further info you wish to see populated. Let me know how that goes. I see you dropped a separate email about a crash, along with a backtrace, thanks for that. I will look into it, ie. maybe you already hinted the above yourself and got to the next stage, and get back to you. Cheers, Paolo On Fri, Jul 05, 2013 at 02:35:15PM +0200, Joan wrote: Hello, I am trying to get pmacct workting to replace softflowd because we'd like to have the as numbers for the networks populated. To accomplish this I am using the script to generate the networks_file from quagga (I had a couple of issues but it's ok now) This is my pmactd.conf config: /etc/pmacct/pmactd.conf daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon aggregate: src_host,dst_host pcap_filter: net 0.0.0.0/0 interface: br0 plugins: nfprobe nfprobe_version: 9 networks_file: /etc/pmacct/networks.lst nfprobe_receiver: 192.168.1.5:2591 http://192.168.1.8:2591 nfprobe_version: 9 And in the flow collector I am checking for the as numbers with nfdump, but the output of srcas/dstas is always 0 nfdump -A srcas -N -M /var/lib/netflow/profiles-data/live/ -o fmt:%sa %fl %byt %pkt %sas %das -R nfcapd.201307051420:nfcapd.201307051425 Did I miss something in the pmacctd config? I don't see anything relevant in the logs. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists nfdump -M /var/lib/netflow/profiles-data/live/ -R nfcapd.201307091410:nfcapd.201307091440 -A srcas,dstas,srcip,dstip Date flow start Duration Src AS Dst AS Src IP Addr Dst IP Addr PacketsBytes bpsBpp Flows 2013-07-09 13:30:18.67914.592 0 0218.94.15.226 123.123.123.1233 120 65 40 1 2013-07-09 14:07:54.345 3.094 0 0 92.81.226.61 123.123.123.1232 96 248 48 1 2013-07-09 14:32:49.080 0.000 0 0 188.165.95.171 123.123.123.1241 440 44 1 2013-07-09 09:20:01.379 18867.828 0 023.123.123.25 224.0.0.6 1473 110892 47 75 1 2013-07-09 13:21:32.957 0.000 0 0 85.237.35.52 123.123.123.1231 600 60 1 2013-07-09 14:26:16.360 0.000 0 0 80.82.64.231 123.123.123.1241 290 29 1 2013-07-09 13:47:01.881 0.000 0 0 186.202.186.28 123.123.123.1241 520 52 1 2013-07-09 09:19:59.525 18878.256 0 0 123.123.123.25 224.0.0.5 1889 151120 64 80 1 2013-07-09 13:28:24.305 0.000 0 0 61.147.103.117 123.123.123.1231 400 40 1 ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Pmacct not adding the as number
I have done the change, I don't have yet any significant amount of flows to analyze, so I'll be back later when I have more information. Thanks a lot for your help, Joan 2013/7/9 Paolo Lucente pa...@pmacct.net Hi Joan, Please add 'pmacctd_as: file' to your config. Actually, in absence of any config directive at this propo, this should be the default setting (if, of course, a networks_file is loaded and we speak pmacctd daemon). Will reproduce your config in lab and see why that would not be happening. Cheers, Paolo On Tue, Jul 09, 2013 at 02:56:30PM +0200, Joan wrote: Hi again, I am reopening this thread again because after upgrading to current 0.14.3 version (which fixes all my crashes) the srcas/dstas data still isn't populated. This is my current config: daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon aggregate: src_host,dst_host,dst_as,src_as interface: br0 plugins: nfprobe networks_file: /etc/pmacct/networks.lst nfprobe_receiver: 192.168.1.5:2591 nfprobe_version: 9 debug : true See the attached document for the dump of the flows that I am doing in the flow collector. And an sample entry in the networks.lst file for one of the matches in the file: 29073,80.82.64.0/24 2013/7/5 Paolo Lucente pa...@pmacct.net xHi Joan, Thanks for explaining the background, it makes sense. To get ASNs info populated you should add src_as and dst_as primitives to your aggregate directive. Same as any further info you wish to see populated. Let me know how that goes. I see you dropped a separate email about a crash, along with a backtrace, thanks for that. I will look into it, ie. maybe you already hinted the above yourself and got to the next stage, and get back to you. Cheers, Paolo On Fri, Jul 05, 2013 at 02:35:15PM +0200, Joan wrote: Hello, I am trying to get pmacct workting to replace softflowd because we'd like to have the as numbers for the networks populated. To accomplish this I am using the script to generate the networks_file from quagga (I had a couple of issues but it's ok now) This is my pmactd.conf config: /etc/pmacct/pmactd.conf daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon aggregate: src_host,dst_host pcap_filter: net 0.0.0.0/0 interface: br0 plugins: nfprobe nfprobe_version: 9 networks_file: /etc/pmacct/networks.lst nfprobe_receiver: 192.168.1.5:2591 http://192.168.1.8:2591 nfprobe_version: 9 And in the flow collector I am checking for the as numbers with nfdump, but the output of srcas/dstas is always 0 nfdump -A srcas -N -M /var/lib/netflow/profiles-data/live/ -o fmt:%sa %fl %byt %pkt %sas %das -R nfcapd.201307051420:nfcapd.201307051425 Did I miss something in the pmacctd config? I don't see anything relevant in the logs. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists nfdump -M /var/lib/netflow/profiles-data/live/ -R nfcapd.201307091410:nfcapd.201307091440 -A srcas,dstas,srcip,dstip Date flow start Duration Src AS Dst AS Src IP Addr Dst IP Addr PacketsBytes bpsBpp Flows 2013-07-09 13:30:18.67914.592 0 0218.94.15.226 123.123.123.1233 120 65 40 1 2013-07-09 14:07:54.345 3.094 0 0 92.81.226.61 123.123.123.1232 96 248 48 1 2013-07-09 14:32:49.080 0.000 0 0 188.165.95.171 123.123.123.1241 440 44 1 2013-07-09 09:20:01.379 18867.828 0 023.123.123.25 224.0.0.6 1473 110892 47 75 1 2013-07-09 13:21:32.957 0.000 0 0 85.237.35.52 123.123.123.1231 600 60 1 2013-07-09 14:26:16.360 0.000 0 0 80.82.64.231 123.123.123.1241 290 29 1 2013-07-09 13:47:01.881 0.000 0 0 186.202.186.28 123.123.123.1241 520 52 1 2013-07-09 09:19:59.525 18878.256 0 0 123.123.123.25 224.0.0.5 1889 151120 64 80 1 2013-07-09 13:28:24.305 0.000 0 0 61.147.103.117 123.123.123.1231 400 40 1 ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Crash in pmacct
I have tried the version in wheezy with the same results as with squeeze, now, I am trying to reproduce the crash with the 0.14.3 downloaded from the site. So far it hasn't crashed, but so far there's only minimal traffic via this router. I'll be back with more info... 2013/7/6 Karl O. Pinc k...@meme.com As an alternative you should consider upgrading to debian wheezy as squeeze will go out of support about 2013-11-04, in 4 months. You'll have to upgrade anyway and this might fix your problem. Wheezy has pmacct 0.14.0. You can get help with any of this for debian using irc chat on the #debian channel of irc.freenode.net. On 07/05/2013 05:39:41 PM, Paolo Lucente wrote: Hi Joan, I can verify the backtrace you provided does not apply to the current (and 0.14.3 release to that matter) code. Also, the issue is related to querying the content of a networks_file - which is a part of the code that got some changes meanwhile. I propose you download/compile 0.14.3 release or CVS code and try again. If these still give troubles please send me privately a new backtrace to inspect. Let me know. Cheers, Paolo On Fri, Jul 05, 2013 at 06:46:21PM +0200, Joan wrote: Hi again, I am experiencing crashes only after a couple of minutes of starting-04 pmacctd. I am on the current squeeze version, but I recompiled from the sources to get non-stripped binaries. After running the process for some minutes the program crashes as usually leaving a nice backtrace. Could you have a look into this and tell me if it's something that was fixed in a newer version? Regards, Joan ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Crash in pmacct
BTW, just found in the changelog for 0.14.1 this: ! fix, net_aggr.c: defining a networks_file configuration directive in conjunction with --enable-ipv6 was causing a SEGVs. This is now solved. That could be the cause for my issue (unless debian backported the fixes) 2013/7/8 Joan aseq...@gmail.com I have tried the version in wheezy with the same results as with squeeze, now, I am trying to reproduce the crash with the 0.14.3 downloaded from the site. So far it hasn't crashed, but so far there's only minimal traffic via this router. I'll be back with more info... 2013/7/6 Karl O. Pinc k...@meme.com As an alternative you should consider upgrading to debian wheezy as squeeze will go out of support about 2013-11-04, in 4 months. You'll have to upgrade anyway and this might fix your problem. Wheezy has pmacct 0.14.0. You can get help with any of this for debian using irc chat on the #debian channel of irc.freenode.net. On 07/05/2013 05:39:41 PM, Paolo Lucente wrote: Hi Joan, I can verify the backtrace you provided does not apply to the current (and 0.14.3 release to that matter) code. Also, the issue is related to querying the content of a networks_file - which is a part of the code that got some changes meanwhile. I propose you download/compile 0.14.3 release or CVS code and try again. If these still give troubles please send me privately a new backtrace to inspect. Let me know. Cheers, Paolo On Fri, Jul 05, 2013 at 06:46:21PM +0200, Joan wrote: Hi again, I am experiencing crashes only after a couple of minutes of starting-04 pmacctd. I am on the current squeeze version, but I recompiled from the sources to get non-stripped binaries. After running the process for some minutes the program crashes as usually leaving a nice backtrace. Could you have a look into this and tell me if it's something that was fixed in a newer version? Regards, Joan ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Crash in pmacct
The wheezy defautl was crashing for me a bit after loading the networks_file (that take about a couple of minutes to load) I was trying to isolate this to open a bug in debian, so at least others are warned. After unsetting the --enable-ipv6 flag and recompile again with debian settings/patches, it seems that it doens't crash anymore. Still I will recompile the 0.14.3 version because I was planning to use the extended format of networks_file for the nexthop feature. 2013/7/8 George-Cristian Bîrzan g...@birzan.org I think I reported that bug, and it was crashing instantly on start, not within minutes. Also, I think that never ended up in a release afair, it was just in trunk. On 8 Jul 2013 13:30, Joan aseq...@gmail.com wrote: BTW, just found in the changelog for 0.14.1 this: ! fix, net_aggr.c: defining a networks_file configuration directive in conjunction with --enable-ipv6 was causing a SEGVs. This is now solved. That could be the cause for my issue (unless debian backported the fixes) 2013/7/8 Joan aseq...@gmail.com I have tried the version in wheezy with the same results as with squeeze, now, I am trying to reproduce the crash with the 0.14.3 downloaded from the site. So far it hasn't crashed, but so far there's only minimal traffic via this router. I'll be back with more info... 2013/7/6 Karl O. Pinc k...@meme.com As an alternative you should consider upgrading to debian wheezy as squeeze will go out of support about 2013-11-04, in 4 months. You'll have to upgrade anyway and this might fix your problem. Wheezy has pmacct 0.14.0. You can get help with any of this for debian using irc chat on the #debian channel of irc.freenode.net. On 07/05/2013 05:39:41 PM, Paolo Lucente wrote: Hi Joan, I can verify the backtrace you provided does not apply to the current (and 0.14.3 release to that matter) code. Also, the issue is related to querying the content of a networks_file - which is a part of the code that got some changes meanwhile. I propose you download/compile 0.14.3 release or CVS code and try again. If these still give troubles please send me privately a new backtrace to inspect. Let me know. Cheers, Paolo On Fri, Jul 05, 2013 at 06:46:21PM +0200, Joan wrote: Hi again, I am experiencing crashes only after a couple of minutes of starting-04 pmacctd. I am on the current squeeze version, but I recompiled from the sources to get non-stripped binaries. After running the process for some minutes the program crashes as usually leaving a nice backtrace. Could you have a look into this and tell me if it's something that was fixed in a newer version? Regards, Joan ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Crash in pmacct
@george, the issue is not the one you reported (that was against 0.14.3cvs) but with an older version. revision 1.16 date: 2012-04-12 14:44:30 +0200; author: paolo; state: Exp; lines: +3 -3; * nfacctd: etype primitive can now be populated from IP_PROTOCOL_VERSION, ie. Field Type #60, in addition to ETHERTYPE, ie. Field Type #256. Should both be present the latter has priority over the former. * fix, net_aggr.c: if --enable-ipv6 is specified, defining a networks_file can cause SEGVs. This is now solved. 2013/7/8 Joan aseq...@gmail.com The wheezy defautl was crashing for me a bit after loading the networks_file (that take about a couple of minutes to load) I was trying to isolate this to open a bug in debian, so at least others are warned. After unsetting the --enable-ipv6 flag and recompile again with debian settings/patches, it seems that it doens't crash anymore. Still I will recompile the 0.14.3 version because I was planning to use the extended format of networks_file for the nexthop feature. 2013/7/8 George-Cristian Bîrzan g...@birzan.org I think I reported that bug, and it was crashing instantly on start, not within minutes. Also, I think that never ended up in a release afair, it was just in trunk. On 8 Jul 2013 13:30, Joan aseq...@gmail.com wrote: BTW, just found in the changelog for 0.14.1 this: ! fix, net_aggr.c: defining a networks_file configuration directive in conjunction with --enable-ipv6 was causing a SEGVs. This is now solved. That could be the cause for my issue (unless debian backported the fixes) 2013/7/8 Joan aseq...@gmail.com I have tried the version in wheezy with the same results as with squeeze, now, I am trying to reproduce the crash with the 0.14.3 downloaded from the site. So far it hasn't crashed, but so far there's only minimal traffic via this router. I'll be back with more info... 2013/7/6 Karl O. Pinc k...@meme.com As an alternative you should consider upgrading to debian wheezy as squeeze will go out of support about 2013-11-04, in 4 months. You'll have to upgrade anyway and this might fix your problem. Wheezy has pmacct 0.14.0. You can get help with any of this for debian using irc chat on the #debian channel of irc.freenode.net. On 07/05/2013 05:39:41 PM, Paolo Lucente wrote: Hi Joan, I can verify the backtrace you provided does not apply to the current (and 0.14.3 release to that matter) code. Also, the issue is related to querying the content of a networks_file - which is a part of the code that got some changes meanwhile. I propose you download/compile 0.14.3 release or CVS code and try again. If these still give troubles please send me privately a new backtrace to inspect. Let me know. Cheers, Paolo On Fri, Jul 05, 2013 at 06:46:21PM +0200, Joan wrote: Hi again, I am experiencing crashes only after a couple of minutes of starting-04 pmacctd. I am on the current squeeze version, but I recompiled from the sources to get non-stripped binaries. After running the process for some minutes the program crashes as usually leaving a nice backtrace. Could you have a look into this and tell me if it's something that was fixed in a newer version? Regards, Joan ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists Karl k...@meme.com Free Software: You don't pay back, you pay forward. -- Robert A. Heinlein ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Pmacct not adding the as number
Hello, I am trying to get pmacct workting to replace softflowd because we'd like to have the as numbers for the networks populated. To accomplish this I am using the script to generate the networks_file from quagga (I had a couple of issues but it's ok now) This is my pmactd.conf config: /etc/pmacct/pmactd.conf daemonize: true pidfile: /var/run/pmacctd.pid syslog: daemon aggregate: src_host,dst_host pcap_filter: net 0.0.0.0/0 interface: br0 plugins: nfprobe nfprobe_version: 9 networks_file: /etc/pmacct/networks.lst nfprobe_receiver: 192.168.1.5:2591 http://192.168.1.8:2591 nfprobe_version: 9 And in the flow collector I am checking for the as numbers with nfdump, but the output of srcas/dstas is always 0 nfdump -A srcas -N -M /var/lib/netflow/profiles-data/live/ -o fmt:%sa %fl %byt %pkt %sas %das -R nfcapd.201307051420:nfcapd.201307051425 Did I miss something in the pmacctd config? I don't see anything relevant in the logs. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Crash in pmacct
Hi again, I am experiencing crashes only after a couple of minutes of starting pmacctd. I am on the current squeeze version, but I recompiled from the sources to get non-stripped binaries. After running the process for some minutes the program crashes as usually leaving a nice backtrace. Could you have a look into this and tell me if it's something that was fixed in a newer version? Regards, Joan pmacct.backtrace Description: Binary data ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
