I upgraded to latest version from github
and the error is no longer showing and it do write to the db
but the problem is that it write the ipv6 src_net and dst_net as '::' so
all ipv6 are the same
tried to understand why it do that but failed (seems it is not able to
understand that the ip belong to /64 subnet)
and I though using network_mask directive is basically telling it that each
ipv6 is part of /64 subnet
but that didnt do it
So here is my question
We have /32 ipv6 subnet and we give each customer one /64 subnet
I want to log the traffic per each /64 subnet out of the /32 subnet that we
have
I can do that using network file but this is unrealistic to write every /64
subnet we have in the file
So is there any easier way to do that ?
Thanks
On Fri, Oct 11, 2019 at 2:41 PM Paolo Lucente <pa...@pmacct.net> wrote:
>
> Hi,
>
> Thank you for the extra insight. Unfortunately this is happening on a
> packaged binary (ie. not self compiled) and very old, ie. > 4 years old
> code. Any chance you can switch to something more current, ie. 1.7.3 or
> master code on GitHub?
>
> Paolo
>
> On Fri, Oct 11, 2019 at 09:30:00AM -0400, moftah moftah wrote:
> > Just noticed i didnt send reply to the group
> >
> > I just enabled both debug and mysql query debug
> >
> > the query sent to mysql is really wrong
> >
> > here is the line before the line with error from sys log
> > Oct 10 20:45:01 ALM02 pmacctd[10723]: DEBUG ( out6/mysql ): 1 VALUES
> > statements sent to the MySQL server.
> > Oct 10 20:45:01 ALM02 pmacctd[10723]: ERROR ( out6/mysql ): Column
> 'ip_src'
> > specified twice
> >
> > Oct 10 20:45:01 ALM02 pmacctd[10722]: DEBUG ( in6/mysql ): 1 VALUES
> > statements sent to the MySQL server.
> > Oct 10 20:45:01 ALM02 pmacctd[10722]: ERROR ( in6/mysql ): Column
> 'ip_dst'
> > specified twice
> >
> > which is not that useful but when checking what query sent to db it looks
> > like this
> >
> >
> > 2019-10-11T01:07:01.018311Z 7 Query UPDATE `acct6_out` SET
> > packets=packets+6460, bytes=bytes+1590631 WHERE ip_src='' AND ip_src='::'
> > AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0'
> > AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0'
> > 2019-10-11T01:07:01.029075Z 8 Query UPDATE `acct6_in` SET
> > packets=packets+9576, bytes=bytes+13235525 WHERE ip_dst='::' AND
> dst_port=0
> > AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND
> > ip_src='0.0.0.0' AND ip_dst='0.0.0.0'
> > 2019-10-11T01:07:01.113495Z 8 Query INSERT INTO `acct6_in`
> > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets,
> > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0',
> > '0.0.0.0', 9576, 13235525)
> > 2019-10-11T01:07:01.117569Z 7 Query INSERT INTO `acct6_out`
> > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src,
> > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0',
> > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 6460, 1590631)
> > 2019-10-11T01:08:01.015998Z 11 Query UPDATE `acct6_in` SET
> > packets=packets+244378, bytes=bytes+335595260 WHERE ip_dst='::' AND
> > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND
> > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0'
> > 2019-10-11T01:08:01.016375Z 12 Query UPDATE `acct6_out` SET
> > packets=packets+164730, bytes=bytes+51500942 WHERE ip_src='' AND
> > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND
> > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND
> > ip_dst='0.0.0.0'
> > 2019-10-11T01:08:01.127378Z 12 Query INSERT INTO `acct6_out`
> > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src,
> > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0',
> > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 164730, 51500942)
> > 2019-10-11T01:08:01.127854Z 11 Query INSERT INTO `acct6_in`
> > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets,
> > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0',
> > '0.0.0.0', 244378, 335595260)
> > 2019-10-11T01:09:01.012185Z 15 Query UPDATE `acct6_in` SET
> > packets=packets+276526, bytes=bytes+387029255 WHERE ip_dst='::' AND
> > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND
> > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0'
> > 2019-10-11T01:09:01.027795Z 16 Query UPDATE `acct6_out` SET
> > packets=packets+177194, bytes=bytes+44384720 WHERE ip_src='' AND
> > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND
> > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND
> > ip_dst='0.0.0.0'
> > 2019-10-11T01:09:01.114225Z 16 Query INSERT INTO `acct6_out`
> > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src,
> > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0',
> > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 177194, 44384720)
> > 2019-10-11T01:09:01.224697Z 15 Query INSERT INTO `acct6_in`
> > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets,
> > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0',
> > '0.0.0.0', 276526, 387029255)
> > 2019-10-11T01:10:01.017359Z 18 Query UPDATE `acct6_in` SET
> > packets=packets+294880, bytes=bytes+412303624 WHERE ip_dst='::' AND
> > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND
> > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0'
> > 2019-10-11T01:10:01.017829Z 20 Query UPDATE `acct6_out` SET
> > packets=packets+186580, bytes=bytes+48111642 WHERE ip_src='' AND
> > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND
> > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND
> > ip_dst='0.0.0.0'
> > 2019-10-11T01:10:01.110120Z 18 Query INSERT INTO `acct6_in`
> > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets,
> > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0',
> > '0.0.0.0', 294880, 412303624)
> > 2019-10-11T01:10:01.146967Z 20 Query INSERT INTO `acct6_out`
> > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src,
> > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0',
> > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 186580, 48111642)
> > 2019-10-11T01:11:01.016057Z 23 Query UPDATE `acct6_out` SET
> > packets=packets+167580, bytes=bytes+44638648 WHERE ip_src='' AND
> > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND
> > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND
> > ip_dst='0.0.0.0'
> > 2019-10-11T01:11:01.016832Z 22 Query UPDATE `acct6_in` SET
> > packets=packets+261858, bytes=bytes+363933056 WHERE ip_dst='::' AND
> > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND
> > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0'
> > 2019-10-11T01:11:01.107406Z 22 Query INSERT INTO `acct6_in`
> > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets,
> > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0',
> > '0.0.0.0', 261858, 363933056)
> > 2019-10-11T01:11:01.133611Z 23 Query INSERT INTO `acct6_out`
> > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src,
> > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0',
> > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 167580, 44638648)
> > 2019-10-11T01:12:01.014452Z 30 Query UPDATE `acct6_in` SET
> > packets=packets+242934, bytes=bytes+335519993 WHERE ip_dst='::' AND
> > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND
> > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0'
> > 2019-10-11T01:12:01.015199Z 29 Query UPDATE `acct6_out` SET
> > packets=packets+162488, bytes=bytes+44937541 WHERE ip_src='' AND
> > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND
> > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND
> > ip_dst='0.0.0.0'
> > 2019-10-11T01:12:01.132584Z 29 Query INSERT INTO `acct6_out`
> > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src,
> > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0',
> > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 162488, 44937541)
> > 2019-10-11T01:12:01.141778Z 30 Query INSERT INTO `acct6_in`
> > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets,
> > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0',
> > '0.0.0.0', 242934, 335519993)
> >
> >
> > do you know why it behave like this
> > the version i have is
> > pmacct -V
> > pmacct, pmacct client 1.5.2 (20150907-00)
> >
> > which is the one by default in ubuntu 18 repo
> >
> >
> > Thanks
> >
> > On Thu, Oct 10, 2019 at 7:29 PM Paolo Lucente <pa...@pmacct.net> wrote:
> >
> > >
> > > Hi,
> > >
> > > Thank you for reporting this. Can show the integral error message you
> > > get back from MySQL? It may give relevant additional info; feel free to
> > > anonimize any confidential data it may contain (ie. IP addresses).
> > >
> > > Paolo
> > >
> > >
> > > On Thu, Oct 10, 2019 at 12:34:12PM -0400, moftah moftah wrote:
> > > > Hi All,
> > > > I have issue in making pmacct aggregate traffic for all ipv6 to be
> per
> > > /64
> > > > not individual ip
> > > >
> > > > I am logging ipv4 and ipv6 and i made special plugin for ipv6 so i
> can
> > > use
> > > > network_mask but it does not work
> > > >
> > > > attached is my config
> > > >
> > > > with this config the error i get is
> > > > column ip_src specified twice
> > > > column ip_dst specified twice
> > > >
> > > > what i am doing wring here
> > > >
> > > > the table is v1 table with added 2 fields
> > > > net_dst
> > > > net_src
> > > >
> > > > can anyone understand why i get column specified twice error
> > > >
> > > >
> > > > Thanks
> > > >
> > > > my config
> > > > aggregate[in]: dst_host
> > > > aggregate[out]: src_host
> > > > aggregate[in6]: dst_net
> > > > aggregate[out6]: src_net
> > > > aggregate_filter[in]: dst net xxx.xxx.xxx.0/22 or dst net
> > > ttt.ttt.ttt.0/22
> > > > aggregate_filter[out]: src net xxx.xxx.xxx.0/22 or src net
> > > ttt.ttt.ttt.0/22
> > > > aggregate_filter[in6]: dst net zzzz:zzzz::/36
> > > > aggregate_filter[out6]: src net zzzz:zzzz::/36
> > > > networks_mask[in6]: 64
> > > > networks_mask[out6]: 64
> > > > interface: ens1f0
> > > > plugins: , mysql[in], mysql[out] , mysql[in6], mysql[out6]
> > > > sql_multi_values: 102400000
> > > > sql_history_roundoff: h
> > > > sql_refresh_time: 60
> > > > sql_table[in]: acct_in
> > > > sql_table[out]: acct_out
> > > > sql_table[in6]: acct6_in
> > > > sql_table[out6]: acct6_out
> > > > !sql_host: localhost
> > > > sql_passwd:aaaaa
> > > > sql_user:aaaaa
> > > > plugin_buffer_size: 10240
> > > > plugin_pipe_size: 2048000
> > > > !sql_table_version: 9
> > >
> > > > _______________________________________________
> > > > pmacct-discussion mailing list
> > > > http://www.pmacct.net/#mailinglists
> > >
> > >
> > > _______________________________________________
> > > pmacct-discussion mailing list
> > > http://www.pmacct.net/#mailinglists
> > >
>
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
