Re: [pmacct-discussion] IPv4 and IPv6 sFlow BGP AS
Hi Paolo, thank you for the answer
root@pmacct:/etc/pmacct# cat bgp_agent.map
bgp_ip=176.**.**.252 ip=0.0.0.0/0filter='ip'
bgp_ip=2001:**:**:1::11 ip=0.0.0.0/0filter='ip6'
Unfortunately, it did not help:-(
Running takes place perfectly
Dec 03 01:07:22 INFO ( default/core ): Trying to (re)load map:
/etc/pmacct/bgp_agent.map
Dec 03 01:07:22 INFO ( default/core ): map '/etc/pmacct/bgp_agent.map'
successfully (re)loaded.
Dec 03 01:07:22 INFO ( default/core/BGP ): maximum BGP peers allowed: 4
Dec 03 01:07:22 INFO ( default/core/BGP ): waiting for BGP data on :::179
Dec 03 01:07:24 INFO ( default/core/BGP ): BGP peers usage: 1/4
Dec 03 01:07:24 INFO ( default/core/BGP ): Capability: MultiProtocol [1]
AFI [1] SAFI [1]
Dec 03 01:07:24 INFO ( default/core/BGP ): Capability: 4-bytes AS [41]
ASN [5**81]
Dec 03 01:07:24 INFO ( default/core/BGP ): [Id: 31.**.**.2] BGP_OPEN:
Asn: 5**81 HoldTime: 240
Dec 03 01:07:27 INFO ( default/core ): waiting for sFlow data on :::6343
Dec 03 01:07:27 INFO ( default/mysql ): cache entries=32771 base cache
memory=11369224 bytes
Dec 03 01:08:01 INFO ( default/mysql ): *** Purging cache - START (PID:
1239) ***
Dec 03 01:08:01 INFO ( default/mysql ): *** Purging cache - END (PID:
1239, QN: 3618/3618, ET: 0) ***
Dec 03 01:08:59 INFO ( default/core/BGP ): BGP peers usage: 2/4
Dec 03 01:08:59 INFO ( default/core/BGP ): Capability: MultiProtocol [1]
AFI [2] SAFI [1]
Dec 03 01:08:59 INFO ( default/core/BGP ): Capability: 4-bytes AS [41]
ASN [5**81]
Dec 03 01:08:59 INFO ( default/core/BGP ): [Id: 176.**.**.97] BGP_OPEN:
Asn: 5**81 HoldTime: 180
But here's the result of a
+--+--+++-+---+--+--+--+-+---+---+-+-+
| agent_id | vlan | as_src | as_dst | ip_src |
ip_dst| src_port | dst_port | ip_proto | packets | bytes |
flows | stamp_inserted | stamp_updated |
+--+--+++-+---+--+--+--+-+---+---+-+-+
|0 | 3855 | 0 | 0 | 2001:4860::1:0:893c |
2001:67c:2d40::47 |0 |0 | ipv6-i | 1 | 214 |
0 | 2016-12-03 01:00:00 | 2016-12-03 01:14:01 |
+--+--+++-+---+--+--+--+-+---+---+-+-+
and
|0 | 3800 | 0 | 0 | 31.43.61.166 |
185.38.12.42 |56911 | 80 | tcp | 1 | 64
| 0 | 2016-12-03 01:00:00 | 2016-12-03 01:08:01 |
Perhaps this will give a little more information to solve the problem
Sfacct version
#sfacctd -V
sFlow Accounting Daemon, sfacctd 1.5.2 (20150907-00)
--build=x86_64-linux-gnu
--prefix=/usr '--includedir=${prefix}/include'
'--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info'
--sysconfdir=/etc
--localstatedir=/var
--disable-silent-rules '--libdir=${prefix}/lib/x86_64-linux-gnu'
'--libexecdir=${prefix}/lib/x86_64-linux-gnu' --disable-maintainer-mode
--disable-dependency-tracking
--enable-mmap
--enable-pgsql
--with-pgsql-includes=/usr/include/postgresql
--enable-mysql
--enable-sqlite3
--enable-ipv6
--enable-v4-mapped
--enable-64bit
--enable-threads
--enable-jansson
--enable-geoip
--enable-rabbitmq
---
Database
create table acct_v6 (
agent_id INT(4) UNSIGNED NOT NULL,
class_id CHAR(16) NOT NULL,
vlan INT(2) UNSIGNED NOT NULL,
as_src INT(4) UNSIGNED NOT NULL,
as_dst INT(4) UNSIGNED NOT NULL,
ip_src CHAR(15) NOT NULL,
ip_dst CHAR(15) NOT NULL,
src_port INT(2) UNSIGNED NOT NULL,
dst_port INT(2) UNSIGNED NOT NULL,
ip_proto CHAR(6) NOT NULL,
packets INT UNSIGNED NOT NULL,
bytes BIGINT UNSIGNED NOT NULL,
flows INT UNSIGNED NOT NULL,
stamp_inserted DATETIME NOT NULL,
stamp_updated DATETIME,
PRIMARY KEY (agent_id, class_id, vlan, as_src, as_dst, ip_src,
ip_dst, src_port, dst_port, ip_proto, stamp_inserted)
);
For suggestions, critics, bugs, contact me: Paolo Lucente
.
03.12.2016 0:03, Paolo Lucente пишет:
Hi Sergey,
I guess what you need is to refine your bgp_agent_map as follows:
bgp_ip=176.**.**.252 ip=0.0.0.0/0 filter='ip'
bgp_ip=2001:**:**:1::11 ip=0.0.0.0/0 filter='ip6'
Let me know if this works for you.
Cheers,
Paolo
On Fri, Dec 02, 2016 at 08:09:07PM +0200, Сергей Горшков wrote:
Hi all.
I need your help because I am in a deadlock
My config
#cat sfacctd.conf
! sfacctd configuration
!
!
!
daemonize: true
interface: any
pidfile:/var/run/sfacctd.pid
syslog: daemon
logfile:/var/log/sfacct.log
!
sfacctd_as_new: bgp
bgp_daemon:
Re: [pmacct-discussion] route distinguisher (RD) looks wired when dump the BGP table.
Hi Alberto,
You may, sure. Although, if the issue is it dumps 32k routes out of 42k,
it may be something else than a performance issue. It smells it will need
more info in order to reproduce it.
Cheers,
Paolo
On Fri, Dec 02, 2016 at 09:55:54PM +0100, Alberto Santos wrote:
> Thx a lot
> I also notice performance issues when dumping the table. Today i have 42 k
> mpls vpn v4 routes but it only dumps 32k
> Do u want me to create another issue?
> BR
> Al
>
> On Dec 2, 2016 18:50, "Paolo Lucente" wrote:
>
> >
> > Hi Alberto,
> >
> > I see you opened an issue on GitHub too about this. To avoid duplications,
> > let me handle this there. I will try to replicate the problem in lab and
> > come back to you. It smells like a bug.
> >
> > Cheers,
> > Paolo
> >
> > On Wed, Nov 30, 2016 at 07:29:14PM +0100, Alberto Santos wrote:
> > > Hi,
> > >
> > > I started playing with pmacct and I noticed that the route distinguisher
> > RD
> > > looks wired when dumping the BGP table in a file. Is there any reason
> > why?
> > >
> > > here it is an example:
> > >
> > > "rd": "1:40.0.17.10:256
> > >
> > > The IP address is inverted, it should be 10.17.0.40 and the 2nd part
> > should
> > > be 1806 instead.
> > > Is this a bug? pls find below the complete output from the file.
> > >
> > > [root@hostname]# cat bgp-10_197_1_114-2016_11_29T20_05_00.txt | grep
> > > 10.12.95.16
> > > {"timestamp": "2016-11-29 20:05:00", "peer_ip_src": "10.17.1.14",
> > > "event_type": "dump", "ip_prefix": "10.12.95.16/28", "bgp_nexthop":
> > > "10.17.1.14", "as_path": "65346", "comms": "65346:39", "ecomms":
> > > "RT:65432:2", "origin": 0, "local_pref": 100, "rd": "1:40.0.17.10:256"}
> > >
> > > BR
> > >
> > > Alberto
> >
> > > ___
> > > pmacct-discussion mailing list
> > > http://www.pmacct.net/#mailinglists
> >
> >
> > ___
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists
> >
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] IPv4 and IPv6 sFlow BGP AS
Hi Sergey, I guess what you need is to refine your bgp_agent_map as follows: bgp_ip=176.**.**.252 ip=0.0.0.0/0 filter='ip' bgp_ip=2001:**:**:1::11 ip=0.0.0.0/0 filter='ip6' Let me know if this works for you. Cheers, Paolo On Fri, Dec 02, 2016 at 08:09:07PM +0200, Сергей Горшков wrote: > Hi all. > > I need your help because I am in a deadlock > > My config > > #cat sfacctd.conf > ! sfacctd configuration > ! > ! > ! > daemonize: true > interface: any > pidfile:/var/run/sfacctd.pid > syslog: daemon > logfile:/var/log/sfacct.log > ! > sfacctd_as_new: bgp > bgp_daemon: true > bgp_agent_map: /etc/pmacct/bgp_agent.map > ! > aggregate: tag,vlan,src_as,dst_as,src_host,dst_host,src_port,dst_port,proto > ! > plugins:mysql > sql_db: pmacct > sql_table_version: 6 > sql_table: acct_v6 > sql_host: localhost > sql_user: * > sql_passwd: * > sql_refresh_time: 60 > sql_optimize_clauses: true > sql_history:1h > sql_history_roundoff: h > sql_locking_style: row > -- > > # cat bgp_agent.map > bgp_ip=176.**.**.252 ip=0.0.0.0/0 > bgp_ip=2001:**:**:1::11 ip=0.0.0.0/0 > -- > In this configuration, all IP6 addresses obtained AS 0 > > select * from acct_v6 where ip_src like '%2001:4860%'; > +--+--+++-+---+--+--+--+-+---+---+-+-+ > | agent_id | vlan | as_src | as_dst | ip_src | > ip_dst| src_port | dst_port | ip_proto | packets | bytes | > flows | stamp_inserted | stamp_updated | > +--+--+++-+---+--+--+--+-+---+---+-+-+ > |0 | 3855 | 0 | 0 | 2001:4860::8:0:8f90 | > 2001:67c:2d40::47 |0 |0 | ipv6-i | 2 | 428 | > 0 | 2016-12-01 11:00:00 | 2016-12-01 11:35:01 | > > IP4 everything is fine > > |0 | 3905 | 15169 | 50581 | 173.194.21.80 | 31.43.60.150 > | 443 |21668 | tcp | 2 | 3044 | 0 | 2016-12-01 > 11:00:00 | 2016-12-01 11:16:01 | > > If BGP neighbors to change their place > > # cat bgp_agent.map > bgp_ip=2001:**:**:1::11 ip=0.0.0.0/0 > bgp_ip=176.**.**.252 ip=0.0.0.0/0 > > Then get IP6 AS number and get IP4 AS0 > How to make sFacct collated both IP4 and IP6 addresses with an > autonomous system number?? > > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] aggregate_filters not working for me
Hi Martin, The traffic appears to be VLAN tagged. The aggregate_filter with 'vlan and src net 172.31.11.0/24' actually works for me and returns traffic. The other does not. I tested against your pcap trace with your config. If the problem at your end persists it could be due, dunno, to the version of the libpcap installed? Last resort, if SSH access to your, system is possible, let's follow up privately - i'd be happy to help you out. Cheers, Paolo On Thu, Dec 01, 2016 at 12:52:39PM +, Miethe, Martin wrote: > Helly everybody, > > we want to set up IP based accountig for a students network. All hosts (> > 5.000) have static IP adresses, so PMACCT seems to be the right software to > use! > To get started and understand sflow and pmacct I'm using a small lab > environment with one laptop connected to a HP switch. sflow is enabled at the > switch access port (laptop). > Now I'd like to have 2 mysql tables (in/out) aggregating the consumed > bandwith per IP on a hourly base. > > Here the pmacct config I am using so far: > === > daemonize: true > interface: ens160 > sfacctd_port: 6343 > sfacctd_ip: 172.31.10.84 > > aggregate[in]: dst_host > aggregate[out]: src_host > !aggregate_filter[in]: dst net 172.31.11.0/24 > !aggregate_filter[out]: vlan and src net 172.31.11.0/24 > > plugins: mysql[in], mysql[out] > sql_history: 1h > sql_history_roundoff: h > sql_host: localhost > sql_db: pmacct > sql_table_version: 6 > sql_passwd: > sql_user: > sql_refresh_time: 60 > sql_table [in]: acct_v6_in > sql_table [out]: acct_v6_out > > sfacctd_renormalize: true > logfile: /home/administrator/sfacctd.log > === > > I made 2 screenshots of the 2 mysql tables (in/out) with samples from the > above config and to be able to go more in depth I attached a packet capture > as well. > https://wetransfer.com/downloads/454b0e7e32f2727d12c53826a65220161201124352/2aa52ff9135a80cbb42a5d9684e359b720161201124352/17e3a7 > > Now I actually want pmacct to only aggregate packets from and to my laptop > (172.31.11.46). I thought aggregate_filter would be the right way to go, but > when I remove the comments, pmacct > will not write any samples to the database. It seems like everything gets > filtered when going with the filters. Am I missing something? > > Thanks a lot in advance! > Martin > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] route distinguisher (RD) looks wired when dump the BGP table.
Thx a lot
I also notice performance issues when dumping the table. Today i have 42 k
mpls vpn v4 routes but it only dumps 32k
Do u want me to create another issue?
BR
Al
On Dec 2, 2016 18:50, "Paolo Lucente" wrote:
>
> Hi Alberto,
>
> I see you opened an issue on GitHub too about this. To avoid duplications,
> let me handle this there. I will try to replicate the problem in lab and
> come back to you. It smells like a bug.
>
> Cheers,
> Paolo
>
> On Wed, Nov 30, 2016 at 07:29:14PM +0100, Alberto Santos wrote:
> > Hi,
> >
> > I started playing with pmacct and I noticed that the route distinguisher
> RD
> > looks wired when dumping the BGP table in a file. Is there any reason
> why?
> >
> > here it is an example:
> >
> > "rd": "1:40.0.17.10:256
> >
> > The IP address is inverted, it should be 10.17.0.40 and the 2nd part
> should
> > be 1806 instead.
> > Is this a bug? pls find below the complete output from the file.
> >
> > [root@hostname]# cat bgp-10_197_1_114-2016_11_29T20_05_00.txt | grep
> > 10.12.95.16
> > {"timestamp": "2016-11-29 20:05:00", "peer_ip_src": "10.17.1.14",
> > "event_type": "dump", "ip_prefix": "10.12.95.16/28", "bgp_nexthop":
> > "10.17.1.14", "as_path": "65346", "comms": "65346:39", "ecomms":
> > "RT:65432:2", "origin": 0, "local_pref": 100, "rd": "1:40.0.17.10:256"}
> >
> > BR
> >
> > Alberto
>
> > ___
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists
>
>
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
>
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
[pmacct-discussion] IPv4 and IPv6 sFlow BGP AS
Hi all. I need your help because I am in a deadlock My config #cat sfacctd.conf ! sfacctd configuration ! ! ! daemonize: true interface: any pidfile:/var/run/sfacctd.pid syslog: daemon logfile:/var/log/sfacct.log ! sfacctd_as_new: bgp bgp_daemon: true bgp_agent_map: /etc/pmacct/bgp_agent.map ! aggregate: tag,vlan,src_as,dst_as,src_host,dst_host,src_port,dst_port,proto ! plugins:mysql sql_db: pmacct sql_table_version: 6 sql_table: acct_v6 sql_host: localhost sql_user: * sql_passwd: * sql_refresh_time: 60 sql_optimize_clauses: true sql_history:1h sql_history_roundoff: h sql_locking_style: row -- # cat bgp_agent.map bgp_ip=176.**.**.252 ip=0.0.0.0/0 bgp_ip=2001:**:**:1::11 ip=0.0.0.0/0 -- In this configuration, all IP6 addresses obtained AS 0 select * from acct_v6 where ip_src like '%2001:4860%'; +--+--+++-+---+--+--+--+-+---+---+-+-+ | agent_id | vlan | as_src | as_dst | ip_src | ip_dst| src_port | dst_port | ip_proto | packets | bytes | flows | stamp_inserted | stamp_updated | +--+--+++-+---+--+--+--+-+---+---+-+-+ |0 | 3855 | 0 | 0 | 2001:4860::8:0:8f90 | 2001:67c:2d40::47 |0 |0 | ipv6-i | 2 | 428 | 0 | 2016-12-01 11:00:00 | 2016-12-01 11:35:01 | IP4 everything is fine |0 | 3905 | 15169 | 50581 | 173.194.21.80 | 31.43.60.150 | 443 |21668 | tcp | 2 | 3044 | 0 | 2016-12-01 11:00:00 | 2016-12-01 11:16:01 | If BGP neighbors to change their place # cat bgp_agent.map bgp_ip=2001:**:**:1::11 ip=0.0.0.0/0 bgp_ip=176.**.**.252 ip=0.0.0.0/0 Then get IP6 AS number and get IP4 AS0 How to make sFacct collated both IP4 and IP6 addresses with an autonomous system number?? ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] route distinguisher (RD) looks wired when dump the BGP table.
Hi Alberto,
I see you opened an issue on GitHub too about this. To avoid duplications,
let me handle this there. I will try to replicate the problem in lab and
come back to you. It smells like a bug.
Cheers,
Paolo
On Wed, Nov 30, 2016 at 07:29:14PM +0100, Alberto Santos wrote:
> Hi,
>
> I started playing with pmacct and I noticed that the route distinguisher RD
> looks wired when dumping the BGP table in a file. Is there any reason why?
>
> here it is an example:
>
> "rd": "1:40.0.17.10:256
>
> The IP address is inverted, it should be 10.17.0.40 and the 2nd part should
> be 1806 instead.
> Is this a bug? pls find below the complete output from the file.
>
> [root@hostname]# cat bgp-10_197_1_114-2016_11_29T20_05_00.txt | grep
> 10.12.95.16
> {"timestamp": "2016-11-29 20:05:00", "peer_ip_src": "10.17.1.14",
> "event_type": "dump", "ip_prefix": "10.12.95.16/28", "bgp_nexthop":
> "10.17.1.14", "as_path": "65346", "comms": "65346:39", "ecomms":
> "RT:65432:2", "origin": 0, "local_pref": 100, "rd": "1:40.0.17.10:256"}
>
> BR
>
> Alberto
> ___
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] stamp_inserted
Hi Jaroslav, Unfortunately not as they are integral part of the sql_history feature (which you need to populate the time-related variables of the tables). As an alternative, only for the 'all' tables where you have the other timestamps, you may disable sql_history and write to a fixed, say, 'router1_in' 'router1_out' tables and, ie. via a sql_trigger script, you can take care yourself of the logics of renaming the tables so to include some time-related variable. Cheers, Paolo On Wed, Nov 30, 2016 at 01:00:02PM +0100, Jaroslav Jirásek wrote: > Hi, I use this scerario: > > sql_refresh_time: 120 > sql_history: 2m > sql_history_roundoff: m > sql_dont_try_update: true > nfacctd_pro_rating: true > > aggregate[router1.all.in]: > src_host,dst_host,proto,src_port,dst_port,timestamp_start,timestamp_end > aggregate[router1.all.out]: > src_host,dst_host,proto,src_port,dst_port,timestamp_start,timestamp_end > aggregate[router1.sums.in]: dst_host > aggregate[router1.sums.out]: src_host > > plugins: > mysql[router1.all.in],mysql[router1.all.out],mysql[router1.sums.in],mysql[router1.sums.out] > > sql_table[router1.all.in]: %Y%m%d_router1_in > sql_table[router1.all.out]: %Y%m%d_router1_out > sql_table[router1.sums.in]: %Y_router1_sums_in > sql_table[router1.sums.out]: %Y_router1_sums_out > > sql_startup_delay[router1.all.in]: 240 > sql_startup_delay[router1.all.out]: 240 > sql_startup_delay[router1.sums.in]: 240 > sql_startup_delay[router1.sums.out]: 240 > > in tables %Y%m%d_router1_in and %Y%m%d_router1_out I have columns > stamp_inserted and stamp_updated, > but I don´t need them, because I aggregate nothing. timestamp_start > and timestamp_end is enough. > In these tables I need to store everything for best accuracy when > finding problems. > > In sums tables I don´t need column stamp_updated. > > Is there any way to not store these columns? > > Thank you, Jaroslav > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
