Hi Paolo, I self compiled pmacct--0.14.2 with just one flag --enable-threads. The same nfacctd.conf file as listed previously in this thread was used and the bgp feed has a different source IP address than the netflow feed, here is what I found:
1) nfacctd.conf with bgp_agent_map configured. nfacctd shuts down as before because "no more plugins active" Here is the debug output: ================================================================== Feb 26 12:42:25 nfacctd[2281]: INFO ( default/core ): Start logging ... Feb 26 12:42:25 nfacctd[2281]: INFO ( default/core ): Trying to (re)load map: /etc/pmacct/agent.map Feb 26 12:42:25 nfacctd[2281]: INFO ( default/core ): map '/etc/pmacct/agent.map' successfully (re)loaded. Feb 26 12:42:25 nfacctd[2281]: INFO ( default/core/BGP ): maximum BGP peers allowed: 100 Feb 26 12:42:25 nfacctd[2281]: INFO ( default/core/BGP ): waiting for BGP data on 192.43.217.2:179 Feb 26 12:42:30 nfacctd[2281]: INFO ( ucar_in/memory ): 112640 bytes are available to address shared memory segment; buffer size is 148 bytes. Feb 26 12:42:30 nfacctd[2281]: INFO ( ucar_in/memory ): Trying to allocate a shared memory segment of 4167680 bytes. Feb 26 12:42:30 nfacctd[2281]: INFO ( ucar_out/memory ): 112640 bytes are available to address shared memory segment; buffer size is 148 bytes. Feb 26 12:42:30 nfacctd[2281]: INFO ( ucar_out/memory ): Trying to allocate a shared memory segment of 4167680 bytes. Feb 26 12:42:30 nfacctd[2283]: OK ( ucar_in/memory ): waiting for data on: '/tmp/pmacct_in.pipe' Feb 26 12:42:30 nfacctd[2284]: OK ( ucar_out/memory ): waiting for data on: '/tmp/pmacct_out.pipe' Feb 26 12:42:30 nfacctd[2281]: INFO ( default/core ): waiting for NetFlow data on 0.0.0.0:9992 Feb 26 12:42:34 nfacctd[2281]: INFO: connection lost to 'ucar_in-memory'; closing connection. Feb 26 12:42:34 nfacctd[2281]: INFO: connection lost to 'ucar_out-memory'; closing connection. Feb 26 12:42:34 nfacctd[2281]: INFO: no more plugins active. Shutting down. ============================================================ 2) nfacctd.conf without bgp_agent_map. nfacctd runs and establishes a bgp session. The memory plug-in shows only one entry with an AS of '0'. This is expected because the source IP address of the bgp feed and the source IP address of the netflow feed are different. 3) If I front end pmacct with a program that allows me to spoof the source IP address of the netflow packets before sending them to pmacct on udp port 9992, and I set that IP address to the same as the source IP address of the bgp feed (verified both using tcpdump), nfacctd runs, but no data is ever returned to the memory plugin, only the column titles appear: <SRC|DST>_AS PACKETS BYTES . Note that the bgp_agent_map configuration directive was NOT active. here is the debug output: ======================================================== Feb 26 13:11:48 nfacctd[3937]: INFO ( default/core ): Start logging ... Feb 26 13:11:48 nfacctd[3937]: INFO ( default/core/BGP ): maximum BGP peers allowed: 100 Feb 26 13:11:48 nfacctd[3937]: INFO ( default/core/BGP ): waiting for BGP data on 192.43.217.2:179 Feb 26 13:11:53 nfacctd[3937]: INFO ( ucar_in/memory ): 112640 bytes are available to address shared memory segment; buffer size is 148 bytes. Feb 26 13:11:53 nfacctd[3937]: INFO ( ucar_in/memory ): Trying to allocate a shared memory segment of 4167680 bytes. Feb 26 13:11:53 nfacctd[3937]: INFO ( ucar_out/memory ): 112640 bytes are available to address shared memory segment; buffer size is 148 bytes. Feb 26 13:11:53 nfacctd[3937]: INFO ( ucar_out/memory ): Trying to allocate a shared memory segment of 4167680 bytes. Feb 26 13:11:53 nfacctd[3939]: OK ( ucar_in/memory ): waiting for data on: '/tmp/pmacct_in.pipe' Feb 26 13:11:53 nfacctd[3937]: INFO ( default/core ): waiting for NetFlow data on 0.0.0.0:9992 Feb 26 13:11:53 nfacctd[3940]: OK ( ucar_out/memory ): waiting for data on: '/tmp/pmacct_out.pipe' Feb 26 13:12:09 nfacctd[3937]: INFO ( default/core/BGP ): BGP peers usage: 1/100 ================================================================ Not sure if this information sheds any light on the problem? Thanks! --paul On 2/25/2013 12:39 PM, Paolo Lucente wrote: > Hi Paul, > > Perfectly agree with your thoughts around the aggregation method and > memory required. Can you please download a tarball from the website, > self-compile and give it a try to that one? Can't really say whether > the issue might be with the debian package. If that does not lead to > anything then it would be good if i can have a brief look myself to > the issue for some debugging. Let me know. > > Cheers, > Paolo > > On Mon, Feb 25, 2013 at 11:36:21AM -0700, paul dial wrote: >> Hi Paolo, >> >> Based on my understanding of pmacct, and the nfacctd configuration >> directives being used, I have defined two memory tables: plugins: >> memory[ucar_in], memory[ucar_out] and I'm using the default >> imt_mem_pools_number: 16 and imt_mem_pools_size: 8192. So each table >> would be 16 x 8192 = 131 kbytes. A "top" report from the server shows: >> Mem: 4059040k total, 1380852k used, 2678188k free, 49032k buffers >> Swap: 7926776k total, 346996k used, 7579780k free, 391112k cached >> >> So I believe there enough memory available to allocate these tables. >> >> I'm hoping to track peer src/dst AS for each member of our RON. Based >> on the filter in place, I should only be looking at one member, and >> would expect a worst case of 40 entries in each table. I don't think >> that would be enough to exceed the allocated memory. >> >> nfacctd is running on a debian 6.0 (squeeze) box. I'm using the pmacct >> 0.14.0-1.1 debian package from "testing" and used apt-get to install >> without any modifications. >> >> Thanks for your response, >> >> --paul >> >> >> On 2/23/2013 2:22 AM, Paolo Lucente wrote: >>> Hi Paul, >>> >>> >From the log it appears the memory plugins bail out and then the core >>> >process >>> closes nicely because it has no more plugins to pass data to. The >>> aggregation >>> method is short so it looks strange but is it possible there is very little >>> memory available for the memory tables? The connection to the bgp_agent_map >>> is precisely what you say: without it you have a single entry; with it you >>> are >>> properly populating the tables. Btw, is it a self-compiled executable? >>> >>> Cheers, >>> Paolo >>> >>> On Fri, Feb 22, 2013 at 02:32:22PM -0700, paul dial wrote: >>>> Hi, >>>> >>>> I'm running pmacct-0.14.1 and attempting to setup a bgp feed. The >>>> netflow feed is being tee'd from another process on the box so all the >>>> netflow packets have a source IP address of 127.0.0.2. The bgp feed is >>>> coming in from one of our routers. Here is a snippet from nfacctd.conf: >>>> >>>> daemonize: true >>>> pidfile: /var/run/nfacctd.pid >>>> syslog: daemon >>>> ! >>>> debug: true >>>> ! >>>> >>>> aggregate[ucar_in]: src_as >>>> aggregate_filter[ucar_in]: dst net 128.117.0.0/16 >>>> aggregate[ucar_out]: dst_as >>>> aggregate_filter[ucar_out]: src net 128.117.0.0/16 >>>> >>>> ! plugin_buffer_size: 1024 >>>> nfacctd_port: 9992 >>>> ! nfacctd_time_secs: true >>>> nfacctd_time_new: true >>>> plugins: memory[ucar_in], memory[ucar_out] >>>> imt_path[ucar_out]: /tmp/pmacct_out.pipe >>>> imt_path[ucar_in]: /tmp/pmacct_in.pipe >>>> networks_file: /etc/pmacct/networks.def >>>> >>>> bgp_daemon: true >>>> !bgp_daemon_msglog: true >>>> bgp_daemon_ip: 192.XXX.XXX.XXX >>>> bgp_daemon_max_peers: 100 >>>> nfacctd_as_new: bgp >>>> bgp_peer_src_as_type: bgp >>>> bgp_src_as_path_type: bgp >>>> bgp_src_local_pref_type: bgp >>>> bgp_src_med_type: bgp >>>> bgp_agent_map: /etc/pmacct/agent.map >>>> >>>> My agent.map looks like this: >>>> id=<ip of bgp source> ip=127.0.0.2 >>>> >>>> When bgp_agent_map config key is used, nfacctd attempts to start but >>>> then dies. Here are the log messages: >>>> =================================================================================================== >>>> Feb 22 12:07:02 testflow nfacctd[1437]: INFO ( default/core ): Start >>>> logging ... >>>> Feb 22 12:07:02 testflow nfacctd[1437]: INFO ( default/core ): Trying to >>>> (re)load map: /etc/pmacct/agent.map >>>> Feb 22 12:07:02 testflow nfacctd[1437]: INFO ( default/core ): map >>>> '/etc/pmacct/agent.map' successfully (re)loaded. >>>> Feb 22 12:07:02 testflow nfacctd[1437]: DEBUG ( default/core/BGP ): 1 >>>> thread(s) initialized >>>> Feb 22 12:07:02 testflow nfacctd[1437]: INFO ( default/core/BGP ): >>>> maximum BGP peers allowed: 100 >>>> Feb 22 12:07:02 testflow nfacctd[1437]: INFO ( default/core/BGP ): >>>> waiting for BGP data on 1xx.xxx.xxx.xxx:179 >>>> Feb 22 12:07:07 testflow nfacctd[1437]: INFO ( ucar_in/memory ): 131070 >>>> bytes are available to address shared memory segment; buffer size is 216 >>>> bytes. >>>> Feb 22 12:07:07 testflow nfacctd[1437]: INFO ( ucar_in/memory ): Trying >>>> to allocate a shared memory segment of 3538728 bytes. >>>> Feb 22 12:07:07 testflow nfacctd[1437]: INFO ( ucar_out/memory ): 131070 >>>> bytes are available to address shared memory segment; buffer size is 216 >>>> bytes. >>>> Feb 22 12:07:07 testflow nfacctd[1437]: INFO ( ucar_out/memory ): Trying >>>> to allocate a shared memory segment of 3538728 bytes. >>>> Feb 22 12:07:07 testflow nfacctd[1441]: DEBUG ( /etc/pmacct/networks.def >>>> ): (networks table IPv4) AS: 0, net: 80750000, mask (bit): ffff0000, >>>> mask (num): 10 >>>> Feb 22 12:07:07 testflow nfacctd[1441]: DEBUG ( /etc/pmacct/networks.def >>>> ): IPv4 Networks Cache successfully created: 99991 entries. >>>> Feb 22 12:07:07 testflow nfacctd[1441]: DEBUG ( /etc/pmacct/networks.def >>>> ): (networks table IPv6) AS: 0, net: 0:0:0:0, mask (bit): 0:0:0:0, mask >>>> (num): 0 >>>> Feb 22 12:07:07 testflow nfacctd[1441]: DEBUG ( /etc/pmacct/networks.def >>>> ): IPv6 Networks Cache successfully created: 32771 entries. >>>> Feb 22 12:07:07 testflow nfacctd[1441]: DEBUG ( ucar_in/memory ): >>>> allocating a new memory segment. >>>> Feb 22 12:07:07 testflow nfacctd[1437]: DEBUG ( /etc/pmacct/networks.def >>>> ): (networks table IPv4) AS: 0, net: 80750000, mask (bit): ffff0000, >>>> mask (num): 10 >>>> Feb 22 12:07:07 testflow nfacctd[1443]: DEBUG ( /etc/pmacct/networks.def >>>> ): (networks table IPv4) AS: 0, net: 80750000, mask (bit): ffff0000, >>>> mask (num): 10 >>>> Feb 22 12:07:07 testflow nfacctd[1437]: DEBUG ( /etc/pmacct/networks.def >>>> ): IPv4 Networks Cache successfully created: 99991 entries. >>>> Feb 22 12:07:07 testflow nfacctd[1443]: DEBUG ( /etc/pmacct/networks.def >>>> ): IPv4 Networks Cache successfully created: 99991 entries. >>>> Feb 22 12:07:07 testflow nfacctd[1443]: DEBUG ( /etc/pmacct/networks.def >>>> ): (networks table IPv6) AS: 0, net: 0:0:0:0, mask (bit): 0:0:0:0, mask >>>> (num): 0 >>>> Feb 22 12:07:07 testflow nfacctd[1437]: DEBUG ( /etc/pmacct/networks.def >>>> ): (networks table IPv6) AS: 0, net: 0:0:0:0, mask (bit): 0:0:0:0, mask >>>> (num): 0 >>>> Feb 22 12:07:07 testflow nfacctd[1437]: DEBUG ( /etc/pmacct/networks.def >>>> ): IPv6 Networks Cache successfully created: 32771 entries. >>>> Feb 22 12:07:07 testflow nfacctd[1443]: DEBUG ( /etc/pmacct/networks.def >>>> ): IPv6 Networks Cache successfully created: 32771 entries. >>>> Feb 22 12:07:07 testflow nfacctd[1437]: INFO ( default/core ): waiting >>>> for NetFlow data on :::9992 >>>> Feb 22 12:07:07 testflow nfacctd[1443]: DEBUG ( ucar_out/memory ): >>>> allocating a new memory segment. >>>> Feb 22 12:07:07 testflow nfacctd[1441]: DEBUG ( ucar_in/memory ): >>>> allocating a new memory segment. >>>> Feb 22 12:07:07 testflow nfacctd[1441]: OK ( ucar_in/memory ): waiting >>>> for data on: '/tmp/pmacct_in.pipe' >>>> Feb 22 12:07:07 testflow nfacctd[1443]: DEBUG ( ucar_out/memory ): >>>> allocating a new memory segment. >>>> Feb 22 12:07:07 testflow nfacctd[1443]: OK ( ucar_out/memory ): waiting >>>> for data on: '/tmp/pmacct_out.pipe' >>>> Feb 22 12:07:07 testflow nfacctd[1443]: DEBUG ( ucar_out/memory ): >>>> Selecting bucket 6355. >>>> Feb 22 12:07:07 testflow nfacctd[1437]: INFO: connection lost to >>>> 'ucar_out-memory'; closing connection. >>>> Feb 22 12:07:13 testflow nfacctd[1441]: DEBUG ( ucar_in/memory ): >>>> Selecting bucket 6355. >>>> Feb 22 12:07:20 testflow nfacctd[1437]: INFO: connection lost to >>>> 'ucar_in-memory'; closing connection. >>>> Feb 22 12:07:20 testflow nfacctd[1437]: INFO: no more plugins active. >>>> Shutting down. >>>> ================================================================================================ >>>> >>>> If I comment out the bgp_agent_map config key, then nfacctd will start, >>>> establish a bgp session, and process netflow data, however, all the AS >>>> numbers are listed as '0' (presumably because the IP address of the bgp >>>> feed doesn't match the IP address of the netflow feed). >>>> >>>> Any thoughts on how to resolve this would be greatly appreciated. >>>> >>>> Thanks, >>>> >>>> --paul >>>> >>>> >>>> _______________________________________________ >>>> pmacct-discussion mailing list >>>> http://www.pmacct.net/#mailinglists >>> _______________________________________________ >>> pmacct-discussion mailing list >>> http://www.pmacct.net/#mailinglists >> >> _______________________________________________ >> pmacct-discussion mailing list >> http://www.pmacct.net/#mailinglists > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
