[pmacct-discussion] peer-AS calculation problem
Hello, I am trying to get nfacctd to calculate peer ASNs, both dst and src. According to the documentation, I have these configuration directives: aggregate: src_as, dst_as, peer_src_as, peer_dst_as, [...] bgp_daemon: true bgp_agent_map: /srv/pmacct/test_nfacctd.bgp_agent_map bgp_peer_src_as_type: map bgp_peer_src_as_map: /srv/pmacct/test_nfacctd.bgp_peer_src_as_map bgp_src_as_path_type: bgp We only have a peering with a route server, which acts as a route reflector for nfacctd. Therefore the contents of the agent map always point to this peering: id=w.x.y.z ip=a.b.c.d id=w.x.y.z ip=:::a.b.c.d The notation above is necessary since we have built pmacct with ipv6. The bgp_peer_src_as_map associates bgp next-hops with peer ASNs. Since netflow is currently only exported from one router to nfacctd, all the records are set to point to this source, regardless where the peering actually happens: id=65001 bgp_nexthop=1.2.3.4 ip=a.b.c.d id=65001 bgp_nexthop=1.2.3.4 ip=:::a.b.c.d id=65002 bgp_nexthop=2.3.4.5 ip=a.b.c.d id=65002 bgp_nexthop=2.3.4.5 ip=:::a.b.c.d I am using this repetitive notation due to ipv6, but I am not sure if it is necessary? Nfacctd is getting netflow v9 non-aggregated feed from a Cisco router (12K, IOS 12.0S). It is actually equivalent to netflow v5 but with the addition of BGP next-hop. The next-hop that is being exported in netflow records is actually mangled due to the fact that next-hop self is used by other routers in their iBGP peerings with the particular router. I have confirmed this by looking into the actual netflow datagrams. On the other hand, this next-hop information is not mangled in the BGP feed nfacctd receives through the route server. I am not sure if this affects nfacctd or, perhaps, if it overrides this information by looking up the next-hop (and perhaps also the dst peer AS) in the BGP RIB? The in-memory table shows that nfacctd has successfully looked up peer_dst_as even for records which contain the the mangled next-hop, so it does not seem to be affected by the above. However the table shows nfacctd has trouble finding the peer_src_as. I have added src_host to the aggregates to help with debugging. Here is a sample: SRC_AS DST_AS SRC_AS_PATH PEER_SRC_AS PEER_DST_AS SRC_IP 825382488253 08248 83.212.132.1 65044 12416504401241 194.177.211.75 47616 835947616020965160.40.50.21 47616 12414761601241 160.40.50.21 47616 33294761603329 160.40.50.21 47616 67994761606799 160.40.50.21 65042 12416504201241 195.251.116.198 65042 67996504206799 195.251.116.198 47616 67994761606799 160.40.50.150 [...] (~2600 records in total) The route server shows: rc# sh ip bgp 83.212.132.1 BGP routing table entry for 83.212.128.0/21 Paths: (1 available, best #1, table Default-IP-Routing-Table) Advertised to non peer-group peers: [...] 8253 194.177.209.182 from [...] Origin IGP, metric 0, localpref 120, valid, internal, best Last update: Tue Nov 24 14:06:07 2009 $ grep 194.177.209.182 test_nfacctd.bgp_peer_src_as_map id=8253 bgp_nexthop=194.177.209.182 ip=a.b.c.d id=8253 bgp_nexthop=194.177.209.182 ip=:::a.b.c.d The same is true for all such records. What is common in all such cases is that the origin and peer src AS always match, i.e. we have a direct peering with the source network. However I don't understand why the peer AS lookup should fail. Could this be a bug? This has been tested with 0.12.0rc3 as well as the latest CVS HEAD (checked out yesterday). Best regards, Z. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] peer-AS calculation problem
Hi Zenon, On Wed, Nov 25, 2009 at 12:59:04PM +0200, Zenon Mousmoulas wrote: I am not sure if this affects nfacctd or, perhaps, if it overrides this information by looking up the next-hop (and perhaps also the dst peer AS) in the BGP RIB? If i'm not mistaken you are not using the 'nfacctd_as_new' configuration directive in your config - or it's not set to 'bgp'. If this is the case, that key causes fields that can be grasped from multiple sources, ie. BGP and NetFlow, like src_as, dst_as, bgp_nexthop to be looked up against BGP. Default value is to look them up in the export protocol (NetFlow, sFlow). Still if this holds, can you give it a try? Let me know. Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] pmacct-contribs 20091125 released
VERSION. 20091125 DESCRIPTION. pmacct is a set of network tools to gather, filter and tag IP traffic; it is able to store collected data either into a DB or a memory table. We see any monitoring, billing or accounting environment as a stack where data are picked from the network, get processed in a meaningful way, and optionally formatted to be user-friendly. In the context of this layered approach we also see pmacct residing at its lowest layer (that is, just upon the network); once collected, often data need to be pulled to upper layers of the environment for a range of applications like time-based reporting, pricing model application, policy enforcement, network thresholds handling, nice output to end-users. pmacct-contribs aims to be over the time a collection, though not exaustive, of scripts and works revolved around or topped over pmacct. README. http://www.pmacct.net/contribs-README DOWNLOAD. http://www.pmacct.net/pmacct-contribs-20091125.tar.gz CHANGELOG. + (Stig Thormodsrud) quagga_gen_as_network.pl: a script to generate a AS number/network file (networks_file) in pmacct format from Quagga. NOTES. README contains the contributions list. New contributions are welcome; support to existing contributions is also appreciated. Questions, critics, suggestions, advices to existing contributions must be sent to respective authors. Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Not save data in DB when exit
Hi. Seems when I make kill INT PID_OF_CORE_PROCESS it down, but plugins do not write to database. I see delay before off for plugins, but not see that they change command line to DB writer. And not see data for period. Can you confirm that bug? I use rc3 My config: daemonize: true pidfile: /var/run/nfacctd.pid syslog: daemon refresh_maps: true nfacctd_port: 8818 plugin_buffer_size: 202400 plugin_pipe_size: 2024 networks_file: /etc/pmacct/networks.list ports_file: /etc/pmacct/ports.list pre_tag_map: /etc/pmacct/pretag.map pre_tag_map_entries: 5 sql_host: 91.206.xxx.30 sql_passwd: xxx nfacctd_time_new: true sql_multi_values: 100 sql_locking_style: row sql_table_version: 4 nfacctd_renormalize: true plugins: mysql[t1], mysql[t2], mysql[t3], mysql[t4] aggregate[t1]: src_host, dst_host, src_port, dst_port, proto aggregate[t2]: tag, tag2 aggregate[t3]: src_host, dst_host, tag aggregate[t4]: tag, tag2 sql_table[t1]: acct_t1 sql_history_roundoff[t1]: h sql_history[t1]: 1h sql_refresh_time[t1]: 3600 sql_dont_try_update[t1]: true sql_recovery_logfile[t1]: /var/lib/pmacct/recovery_log_t1 sql_table[t2]: acct_t2 sql_history_roundoff[t2]: h sql_history[t2]: 1h sql_refresh_time[t2]: 3600 sql_dont_try_update[t2]: true sql_recovery_logfile[t2]: /var/lib/pmacct/recovery_log_t2 sql_table[t3]: acct_t3 sql_history_roundoff[t3]: d sql_history[t3]: 1d sql_refresh_time[t3]: 3600 sql_dont_try_update[t3]: false sql_recovery_logfile[t3]: /var/lib/pmacct/recovery_log_t3 sql_table[t4]: acct_t4 sql_history_roundoff[t4]: d sql_history[t4]: 1d sql_refresh_time[t4]: 3600 sql_dont_try_update[t4]: false sql_recovery_logfile[t4]: /var/lib/pmacct/recovery_log_t4 -- WBR, Dubrovskiy Vyacheslav smime.p7s Description: S/MIME Cryptographic Signature ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] peer-AS calculation problem
Hi,
On 25 Νοε 2009, at 5:46 ΜΜ, Paolo Lucente wrote:
On Wed, Nov 25, 2009 at 12:59:04PM +0200, Zenon Mousmoulas wrote:
I am not sure if this affects nfacctd or, perhaps, if it overrides
this
information by looking up the next-hop (and perhaps also the dst
peer AS)
in the BGP RIB?
If i'm not mistaken you are not using the 'nfacctd_as_new'
configuration
directive in your config - or it's not set to 'bgp'. If this is the
case,
that key causes fields that can be grasped from multiple sources,
ie. BGP
and NetFlow, like src_as, dst_as, bgp_nexthop to be looked up
against BGP.
Default value is to look them up in the export protocol (NetFlow,
sFlow).
Still if this holds, can you give it a try?
I was under the impression that 'nfacctd_as_new: bgp' would cause
nfacctd to lookup ASNs even though the origin ASN is already exported
in netflow datagrams; this is something I was trying to avoid.
In any case, I enabled the directive. Comparing the results, there
were a few cases where nfacctd recorded a different dst ASN than what
the AS path shows, when this setting is left to the default value
(false):
DST_AS AS_PATH
97 20965_3549_6789_6789_196705
684920965_1299_6849_{6877}
36619 20965_3549_36625
24326 20965_3549_2914_4651_45758
121 20965_3549_3257_35007_196729
32468 20965_3549_3561_6428_{32468}
532 20965_3549_7738_262676
196 20965_3549_9002_40965_196804
886520965_3549_3356_13293_29414
34169 20965_3549_3257_35007
Some of these are probably 4-byte ASNs obviously exported wrong in
netflow datagrams. When nfacctd_as_new is set to bgp, it doesn't get
these wrong.
So it seems that not trusting netflow records for these fields may
be a good idea after all. Thanks!
Source peer ASN calculation does not seem to be affected by this
setting, however. Nfacctd still misses it somehow.
Cheers,
Z.
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Not save data in DB when exit
Hi Slava, On Wed, Nov 25, 2009 at 09:04:24PM +0200, Slava Dubrovskiy wrote: Seems when I make kill INT PID_OF_CORE_PROCESS it down, but plugins do not write to database. I see delay before off for plugins, but not see that they change command line to DB writer. And not see data for period. Can you confirm that bug? You should send a SIGINT to the plugins you want to write to the database, not only to the core process (just wondering if it's written this way in any part of the documentation). A 'killall INT pmacctd' should do it; or if you need better granularity use the 'pidfile' directive to be able to retrieve the PID for the plugins aswell. Let me know how it goes. Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
