Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Thank you Paolo, I see I can use aggregation filters also. So I guess will find a way to implement what is needed without having a convoluted configuration file. cheers, Alex On Thu, Feb 27, 2020 at 12:24 PM Paolo Lucente wrote: > > Hi Alex, > > Ack. The other way you could "filter" out is with a networks_file: in > there you specify the network(s) you are interested in following the > example here: > > https://github.com/pmacct/pmacct/blob/master/examples/networks.lst.example > > In the simplest case, you just want to list networks of interest one per > line. Then in the config you want to set 'networks_file_filter: true' as > well. This is kind-of filtering: networks / IPs not of interest will be > just zeroed out and rolled up as a 0.0.0.0 src_host / dst_host. > > Paolo > > On Wed, Feb 26, 2020 at 11:32:31AM +0200, Alex K wrote: > > Hi Paolo, > > > > On Tue, Feb 25, 2020 at 6:41 PM Paolo Lucente wrote: > > > > > > > > Hi Alex, > > > > > > Thanks for your feedback. I see you did run "tcpdump -n -vv -i nflog:1" > > > which is equivalent to run uacctd without any filters; as you may know, > > > you can append a BPF-style filter to the tcpdump command-line, > precisely > > > as you express it in pre_tag_map. Can you give that a try and see if > you > > > get any luck? > > > > > Bad luck... I get: > > tcpdump -nvv -i nflog:1 src net 192.168.28.0/24 > > tcpdump: NFLOG link-layer type filtering not implemented > > It seems that filtering at nflog interface is not supported. > > Running tcpdump -nvv -i eth0 src net 192.168.28.0/24 does capture > traffic > > normally. > > Is there any other way I could apply some filtering with uacctd? I need > to > > use uacctd since I get all the pre-nat, post-nat details of the flows, so > > as to account traffic at the WAN interfaces with the real source details. > > > > > > > My expextation is: if something does not work with pre_tag_map, it > > > should also not work with tcpdump; if you work out a filter to work > > > against tcpdump, that should work in pre_tag_map as well. Any > disconnect > > > among the two may bring the scent of a bug. > > > > > > Paolo > > > > > > On Tue, Feb 25, 2020 at 11:20:21AM +0200, Alex K wrote: > > > > Here is the output when running in debug mode: > > > > > > > > INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon, > uacctd > > > > (20200222-01) > > > > INFO ( default/core ): '--prefix=/usr' '--enable-mysql' > '--enable-nflog' > > > > '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' > > > > '--enable-bmp-bins' '--enable-st-bins' > > > > INFO ( default/core ): Reading configuration file > > > > '/root/pmacct/uacctd2.conf'. > > > > INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes > > > > plugin_buffer_size=280 bytes > > > > INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes > > > > target=117024 bytes > > > > INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes > > > > plugin_buffer_size=280 bytes > > > > INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes > > > > target=117024 bytes > > > > INFO ( print_wan0_in/print ): cache entries=16411 base cache > > > > memory=54878384 bytes > > > > INFO ( default/core ): [pretag2.map] (re)loading map. > > > > INFO ( print_wan0_out/print ): cache entries=16411 base cache > > > > memory=54878384 bytes > > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > > > INFO ( default/core ): [pretag2.map] (re)loading map. > > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > > > INFO ( default/core ): [pretag2.map] (re)loading map. > > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > > > INFO ( default/core ): Successfully connected Netlink NFLOG socket > > > > > > > > It doesn't seem to have any issues loading the maps, though it is not > > > > collecting anything. When capturing with tcpdump I see packets going > > > > through: > > > > > > > > tcpdump -n -vv -i nflog:1 > > > > 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF], > > > proto > > > > ICMP (1), length 84) > > > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1, > length > > > 64 > > > > 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], > proto > > > > ICMP (1), length 84) > > > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1, > length 64 > > > > 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF], > > > proto > > > > ICMP (1), length 84) > > > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2, > length > > > 64 > > > > 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], > proto > > > > ICMP (1), length 84) > > > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2, > length 64 > > > > > > > > The pmacct version I am running is latest master. > > > > Thank you for your assistance. > > > > > > > > Alex > > > > > > > > > > > > On Mon, Feb 24, 2020 at 6:20 PM Alex K > wrote: > >
Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Hi Alex, Ack. The other way you could "filter" out is with a networks_file: in there you specify the network(s) you are interested in following the example here: https://github.com/pmacct/pmacct/blob/master/examples/networks.lst.example In the simplest case, you just want to list networks of interest one per line. Then in the config you want to set 'networks_file_filter: true' as well. This is kind-of filtering: networks / IPs not of interest will be just zeroed out and rolled up as a 0.0.0.0 src_host / dst_host. Paolo On Wed, Feb 26, 2020 at 11:32:31AM +0200, Alex K wrote: > Hi Paolo, > > On Tue, Feb 25, 2020 at 6:41 PM Paolo Lucente wrote: > > > > > Hi Alex, > > > > Thanks for your feedback. I see you did run "tcpdump -n -vv -i nflog:1" > > which is equivalent to run uacctd without any filters; as you may know, > > you can append a BPF-style filter to the tcpdump command-line, precisely > > as you express it in pre_tag_map. Can you give that a try and see if you > > get any luck? > > > Bad luck... I get: > tcpdump -nvv -i nflog:1 src net 192.168.28.0/24 > tcpdump: NFLOG link-layer type filtering not implemented > It seems that filtering at nflog interface is not supported. > Running tcpdump -nvv -i eth0 src net 192.168.28.0/24 does capture traffic > normally. > Is there any other way I could apply some filtering with uacctd? I need to > use uacctd since I get all the pre-nat, post-nat details of the flows, so > as to account traffic at the WAN interfaces with the real source details. > > > > My expextation is: if something does not work with pre_tag_map, it > > should also not work with tcpdump; if you work out a filter to work > > against tcpdump, that should work in pre_tag_map as well. Any disconnect > > among the two may bring the scent of a bug. > > > > Paolo > > > > On Tue, Feb 25, 2020 at 11:20:21AM +0200, Alex K wrote: > > > Here is the output when running in debug mode: > > > > > > INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon, uacctd > > > (20200222-01) > > > INFO ( default/core ): '--prefix=/usr' '--enable-mysql' '--enable-nflog' > > > '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' > > > '--enable-bmp-bins' '--enable-st-bins' > > > INFO ( default/core ): Reading configuration file > > > '/root/pmacct/uacctd2.conf'. > > > INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes > > > plugin_buffer_size=280 bytes > > > INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes > > > target=117024 bytes > > > INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes > > > plugin_buffer_size=280 bytes > > > INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes > > > target=117024 bytes > > > INFO ( print_wan0_in/print ): cache entries=16411 base cache > > > memory=54878384 bytes > > > INFO ( default/core ): [pretag2.map] (re)loading map. > > > INFO ( print_wan0_out/print ): cache entries=16411 base cache > > > memory=54878384 bytes > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > > INFO ( default/core ): [pretag2.map] (re)loading map. > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > > INFO ( default/core ): [pretag2.map] (re)loading map. > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > > INFO ( default/core ): Successfully connected Netlink NFLOG socket > > > > > > It doesn't seem to have any issues loading the maps, though it is not > > > collecting anything. When capturing with tcpdump I see packets going > > > through: > > > > > > tcpdump -n -vv -i nflog:1 > > > 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF], > > proto > > > ICMP (1), length 84) > > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1, length > > 64 > > > 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto > > > ICMP (1), length 84) > > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1, length 64 > > > 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF], > > proto > > > ICMP (1), length 84) > > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2, length > > 64 > > > 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto > > > ICMP (1), length 84) > > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2, length 64 > > > > > > The pmacct version I am running is latest master. > > > Thank you for your assistance. > > > > > > Alex > > > > > > > > > On Mon, Feb 24, 2020 at 6:20 PM Alex K wrote: > > > > > > > Hi Paolo, > > > > > > > > On Sat, Feb 22, 2020 at 4:18 PM Paolo Lucente > > wrote: > > > > > > > >> > > > >> Hi Alex, > > > >> > > > >> Is it possible with the new setup - the one where pre_tag_map does not > > > >> match anything - the traffic is VLAN-tagged (or MPLS-labelled)? If so, > > > >> you should adjust filters accordingly and add 'vlan and', ie. "vlan > > and > > > >> src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24". > > > >> >
Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Hi Paolo, On Tue, Feb 25, 2020 at 6:41 PM Paolo Lucente wrote: > > Hi Alex, > > Thanks for your feedback. I see you did run "tcpdump -n -vv -i nflog:1" > which is equivalent to run uacctd without any filters; as you may know, > you can append a BPF-style filter to the tcpdump command-line, precisely > as you express it in pre_tag_map. Can you give that a try and see if you > get any luck? > Bad luck... I get: tcpdump -nvv -i nflog:1 src net 192.168.28.0/24 tcpdump: NFLOG link-layer type filtering not implemented It seems that filtering at nflog interface is not supported. Running tcpdump -nvv -i eth0 src net 192.168.28.0/24 does capture traffic normally. Is there any other way I could apply some filtering with uacctd? I need to use uacctd since I get all the pre-nat, post-nat details of the flows, so as to account traffic at the WAN interfaces with the real source details. > My expextation is: if something does not work with pre_tag_map, it > should also not work with tcpdump; if you work out a filter to work > against tcpdump, that should work in pre_tag_map as well. Any disconnect > among the two may bring the scent of a bug. > > Paolo > > On Tue, Feb 25, 2020 at 11:20:21AM +0200, Alex K wrote: > > Here is the output when running in debug mode: > > > > INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon, uacctd > > (20200222-01) > > INFO ( default/core ): '--prefix=/usr' '--enable-mysql' '--enable-nflog' > > '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' > > '--enable-bmp-bins' '--enable-st-bins' > > INFO ( default/core ): Reading configuration file > > '/root/pmacct/uacctd2.conf'. > > INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes > > plugin_buffer_size=280 bytes > > INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes > > target=117024 bytes > > INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes > > plugin_buffer_size=280 bytes > > INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes > > target=117024 bytes > > INFO ( print_wan0_in/print ): cache entries=16411 base cache > > memory=54878384 bytes > > INFO ( default/core ): [pretag2.map] (re)loading map. > > INFO ( print_wan0_out/print ): cache entries=16411 base cache > > memory=54878384 bytes > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > INFO ( default/core ): [pretag2.map] (re)loading map. > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > INFO ( default/core ): [pretag2.map] (re)loading map. > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > > INFO ( default/core ): Successfully connected Netlink NFLOG socket > > > > It doesn't seem to have any issues loading the maps, though it is not > > collecting anything. When capturing with tcpdump I see packets going > > through: > > > > tcpdump -n -vv -i nflog:1 > > 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF], > proto > > ICMP (1), length 84) > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1, length > 64 > > 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto > > ICMP (1), length 84) > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1, length 64 > > 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF], > proto > > ICMP (1), length 84) > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2, length > 64 > > 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto > > ICMP (1), length 84) > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2, length 64 > > > > The pmacct version I am running is latest master. > > Thank you for your assistance. > > > > Alex > > > > > > On Mon, Feb 24, 2020 at 6:20 PM Alex K wrote: > > > > > Hi Paolo, > > > > > > On Sat, Feb 22, 2020 at 4:18 PM Paolo Lucente > wrote: > > > > > >> > > >> Hi Alex, > > >> > > >> Is it possible with the new setup - the one where pre_tag_map does not > > >> match anything - the traffic is VLAN-tagged (or MPLS-labelled)? If so, > > >> you should adjust filters accordingly and add 'vlan and', ie. "vlan > and > > >> src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24". > > >> > > > The traffic is not VLAN or MPLS. It is simple one. I confirm I can > collect > > > traffic when removing the pretag directives. Also when stopping > uacctd, I > > > can capture traffic at nflog:1 interface. > > > I simplified the configuration as below: > > > > > > ! > > > daemonize: true > > > promisc: false > > > uacctd_group: 1 > > > ! > > > pre_tag_map: pretag2.map > > > pre_tag_filter[print_wan0_in]: 1 > > > pre_tag_filter[print_wan0_out]: 2 > > > ! > > > !- > > > plugins: print[print_wan0_in], print[print_wan0_out] > > > print_refresh_time: 10 > > > print_history: 15m > > > print_output_file_append: true > > > ! > > > print_output[print_wan0_in]: csv > > > print_output[print_wan0_out]: csv > > > print_output_file[print_wan0_in]:
Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Hi Alex, Thanks for your feedback. I see you did run "tcpdump -n -vv -i nflog:1" which is equivalent to run uacctd without any filters; as you may know, you can append a BPF-style filter to the tcpdump command-line, precisely as you express it in pre_tag_map. Can you give that a try and see if you get any luck? My expextation is: if something does not work with pre_tag_map, it should also not work with tcpdump; if you work out a filter to work against tcpdump, that should work in pre_tag_map as well. Any disconnect among the two may bring the scent of a bug. Paolo On Tue, Feb 25, 2020 at 11:20:21AM +0200, Alex K wrote: > Here is the output when running in debug mode: > > INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon, uacctd > (20200222-01) > INFO ( default/core ): '--prefix=/usr' '--enable-mysql' '--enable-nflog' > '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' > '--enable-bmp-bins' '--enable-st-bins' > INFO ( default/core ): Reading configuration file > '/root/pmacct/uacctd2.conf'. > INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes > plugin_buffer_size=280 bytes > INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes > target=117024 bytes > INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes > plugin_buffer_size=280 bytes > INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes > target=117024 bytes > INFO ( print_wan0_in/print ): cache entries=16411 base cache > memory=54878384 bytes > INFO ( default/core ): [pretag2.map] (re)loading map. > INFO ( print_wan0_out/print ): cache entries=16411 base cache > memory=54878384 bytes > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > INFO ( default/core ): [pretag2.map] (re)loading map. > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > INFO ( default/core ): [pretag2.map] (re)loading map. > INFO ( default/core ): [pretag2.map] map successfully (re)loaded. > INFO ( default/core ): Successfully connected Netlink NFLOG socket > > It doesn't seem to have any issues loading the maps, though it is not > collecting anything. When capturing with tcpdump I see packets going > through: > > tcpdump -n -vv -i nflog:1 > 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF], proto > ICMP (1), length 84) > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1, length 64 > 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto > ICMP (1), length 84) > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1, length 64 > 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF], proto > ICMP (1), length 84) > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2, length 64 > 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto > ICMP (1), length 84) > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2, length 64 > > The pmacct version I am running is latest master. > Thank you for your assistance. > > Alex > > > On Mon, Feb 24, 2020 at 6:20 PM Alex K wrote: > > > Hi Paolo, > > > > On Sat, Feb 22, 2020 at 4:18 PM Paolo Lucente wrote: > > > >> > >> Hi Alex, > >> > >> Is it possible with the new setup - the one where pre_tag_map does not > >> match anything - the traffic is VLAN-tagged (or MPLS-labelled)? If so, > >> you should adjust filters accordingly and add 'vlan and', ie. "vlan and > >> src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24". > >> > > The traffic is not VLAN or MPLS. It is simple one. I confirm I can collect > > traffic when removing the pretag directives. Also when stopping uacctd, I > > can capture traffic at nflog:1 interface. > > I simplified the configuration as below: > > > > ! > > daemonize: true > > promisc: false > > uacctd_group: 1 > > ! > > pre_tag_map: pretag2.map > > pre_tag_filter[print_wan0_in]: 1 > > pre_tag_filter[print_wan0_out]: 2 > > ! > > !- > > plugins: print[print_wan0_in], print[print_wan0_out] > > print_refresh_time: 10 > > print_history: 15m > > print_output_file_append: true > > ! > > print_output[print_wan0_in]: csv > > print_output[print_wan0_out]: csv > > print_output_file[print_wan0_in]: traffic-wan0-in.csv > > print_output_file[print_wan0_out]: traffic-wan0-out.csv > > ! > > aggregate[print_wan0_in]: tag, src_host, dst_host, src_port, dst_port, > > proto > > aggregate[print_wan0_out]: tag, src_host, dst_host, src_port, dst_port, > > proto > > ! > > > > with pretag2.map > > set_tag=1 filter='src net 192.168.28.0/24' > > set_tag=2 filter='dst net 192.168.28.0/24' > > > > As soon as I enable the pretag directives as below, I do not see any > > traffic being collected from uacctd at NFLOG goup 1 > > > > pre_tag_map: pretag2.map > > pre_tag_filter[print_wan0_in]: 1 > > pre_tag_filter[print_wan0_out]: 2 > > > > I am running pmacct 1.7.4. > > > > > >> Paolo > >> > >> On Fri, Feb 21, 2020 at 01:04:25PM +0200, Alex K wrote: > >> > Working further on this,
Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Here is the output when running in debug mode: INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon, uacctd (20200222-01) INFO ( default/core ): '--prefix=/usr' '--enable-mysql' '--enable-nflog' '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins' INFO ( default/core ): Reading configuration file '/root/pmacct/uacctd2.conf'. INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes plugin_buffer_size=280 bytes INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes target=117024 bytes INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes plugin_buffer_size=280 bytes INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes target=117024 bytes INFO ( print_wan0_in/print ): cache entries=16411 base cache memory=54878384 bytes INFO ( default/core ): [pretag2.map] (re)loading map. INFO ( print_wan0_out/print ): cache entries=16411 base cache memory=54878384 bytes INFO ( default/core ): [pretag2.map] map successfully (re)loaded. INFO ( default/core ): [pretag2.map] (re)loading map. INFO ( default/core ): [pretag2.map] map successfully (re)loaded. INFO ( default/core ): [pretag2.map] (re)loading map. INFO ( default/core ): [pretag2.map] map successfully (re)loaded. INFO ( default/core ): Successfully connected Netlink NFLOG socket It doesn't seem to have any issues loading the maps, though it is not collecting anything. When capturing with tcpdump I see packets going through: tcpdump -n -vv -i nflog:1 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1, length 64 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto ICMP (1), length 84) 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1, length 64 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF], proto ICMP (1), length 84) 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2, length 64 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto ICMP (1), length 84) 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2, length 64 The pmacct version I am running is latest master. Thank you for your assistance. Alex On Mon, Feb 24, 2020 at 6:20 PM Alex K wrote: > Hi Paolo, > > On Sat, Feb 22, 2020 at 4:18 PM Paolo Lucente wrote: > >> >> Hi Alex, >> >> Is it possible with the new setup - the one where pre_tag_map does not >> match anything - the traffic is VLAN-tagged (or MPLS-labelled)? If so, >> you should adjust filters accordingly and add 'vlan and', ie. "vlan and >> src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24". >> > The traffic is not VLAN or MPLS. It is simple one. I confirm I can collect > traffic when removing the pretag directives. Also when stopping uacctd, I > can capture traffic at nflog:1 interface. > I simplified the configuration as below: > > ! > daemonize: true > promisc: false > uacctd_group: 1 > ! > pre_tag_map: pretag2.map > pre_tag_filter[print_wan0_in]: 1 > pre_tag_filter[print_wan0_out]: 2 > ! > !- > plugins: print[print_wan0_in], print[print_wan0_out] > print_refresh_time: 10 > print_history: 15m > print_output_file_append: true > ! > print_output[print_wan0_in]: csv > print_output[print_wan0_out]: csv > print_output_file[print_wan0_in]: traffic-wan0-in.csv > print_output_file[print_wan0_out]: traffic-wan0-out.csv > ! > aggregate[print_wan0_in]: tag, src_host, dst_host, src_port, dst_port, > proto > aggregate[print_wan0_out]: tag, src_host, dst_host, src_port, dst_port, > proto > ! > > with pretag2.map > set_tag=1 filter='src net 192.168.28.0/24' > set_tag=2 filter='dst net 192.168.28.0/24' > > As soon as I enable the pretag directives as below, I do not see any > traffic being collected from uacctd at NFLOG goup 1 > > pre_tag_map: pretag2.map > pre_tag_filter[print_wan0_in]: 1 > pre_tag_filter[print_wan0_out]: 2 > > I am running pmacct 1.7.4. > > >> Paolo >> >> On Fri, Feb 21, 2020 at 01:04:25PM +0200, Alex K wrote: >> > Working further on this, it seems that for pmacct is sufficient to >> filter >> > traffic using only the pre_tag_filter, thus no need for the aggregation >> > filters. >> > The issue with this setup though is that I loose the information of the >> > pre_nat source IP address when monitoring at the WAN interfaces. Due to >> > this I am switching to uacctd as following: >> > >> > ! >> > daemonize: true >> > promisc: false >> > uacctd_group: 1 >> > !networks_file: networks.lst >> > !ports_file: ports.lst >> > ! >> > pre_tag_map: pretag2.map >> > pre_tag_filter[print_wan0_in]: 1 >> > pre_tag_filter[print_wan0_out]: 2 >> > pre_tag_filter[wan0_in]: 1 >> > pre_tag_filter[wan0_out]: 2 >> > ! >> > plugins: print[print_wan0_in], print[print_wan0_out], mysql[wan0_in], >> > mysql[wan0_out] >> > plugin_pipe_size[wan0_in]: 1024000 >> > plugin_pipe_size[wan0_out]: 1024000 >> >
Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Hi Paolo, On Sat, Feb 22, 2020 at 4:18 PM Paolo Lucente wrote: > > Hi Alex, > > Is it possible with the new setup - the one where pre_tag_map does not > match anything - the traffic is VLAN-tagged (or MPLS-labelled)? If so, > you should adjust filters accordingly and add 'vlan and', ie. "vlan and > src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24". > The traffic is not VLAN or MPLS. It is simple one. I confirm I can collect traffic when removing the pretag directives. Also when stopping uacctd, I can capture traffic at nflog:1 interface. I simplified the configuration as below: ! daemonize: true promisc: false uacctd_group: 1 ! pre_tag_map: pretag2.map pre_tag_filter[print_wan0_in]: 1 pre_tag_filter[print_wan0_out]: 2 ! !- plugins: print[print_wan0_in], print[print_wan0_out] print_refresh_time: 10 print_history: 15m print_output_file_append: true ! print_output[print_wan0_in]: csv print_output[print_wan0_out]: csv print_output_file[print_wan0_in]: traffic-wan0-in.csv print_output_file[print_wan0_out]: traffic-wan0-out.csv ! aggregate[print_wan0_in]: tag, src_host, dst_host, src_port, dst_port, proto aggregate[print_wan0_out]: tag, src_host, dst_host, src_port, dst_port, proto ! with pretag2.map set_tag=1 filter='src net 192.168.28.0/24' set_tag=2 filter='dst net 192.168.28.0/24' As soon as I enable the pretag directives as below, I do not see any traffic being collected from uacctd at NFLOG goup 1 pre_tag_map: pretag2.map pre_tag_filter[print_wan0_in]: 1 pre_tag_filter[print_wan0_out]: 2 I am running pmacct 1.7.4. > Paolo > > On Fri, Feb 21, 2020 at 01:04:25PM +0200, Alex K wrote: > > Working further on this, it seems that for pmacct is sufficient to filter > > traffic using only the pre_tag_filter, thus no need for the aggregation > > filters. > > The issue with this setup though is that I loose the information of the > > pre_nat source IP address when monitoring at the WAN interfaces. Due to > > this I am switching to uacctd as following: > > > > ! > > daemonize: true > > promisc: false > > uacctd_group: 1 > > !networks_file: networks.lst > > !ports_file: ports.lst > > ! > > pre_tag_map: pretag2.map > > pre_tag_filter[print_wan0_in]: 1 > > pre_tag_filter[print_wan0_out]: 2 > > pre_tag_filter[wan0_in]: 1 > > pre_tag_filter[wan0_out]: 2 > > ! > > plugins: print[print_wan0_in], print[print_wan0_out], mysql[wan0_in], > > mysql[wan0_out] > > plugin_pipe_size[wan0_in]: 1024000 > > plugin_pipe_size[wan0_out]: 1024000 > > print_refresh_time: 10 > > print_history: 15m > > print_output_file_append: true > > ! > > print_output[print_wan0_in]: csv > > print_output_file[print_wan0_in]: in_traffic.csv > > print_output[print_wan0_out]: csv > > print_output_file[print_wan0_out]: out_traffic.csv > > ! > > aggregate[print_wan0_in]: dst_host, src_port, dst_port, proto > > aggregate[print_wan0_out]: src_host, src_port, dst_port, proto > > ! > > sql_table[wan0_in]: traffic_wan0_in_%Y%m%d_%H%M > > sql_table[wan0_out]: traffic_wan0_out_%Y%m%d_%H%M > > ! > > sql_table_schema[wan0_in]: traffic_wan0_in.schema > > sql_table_schema[wan0_out]: traffic_wan0_out.schema > > ! > > sql_host: localhost > > sql_db : uacct > > sql_user : uacct > > sql_passwd: uacct > > sql_refresh_time: 30 > > sql_optimize_clauses: true > > sql_history : 24h > > sql_history_roundoff: mhd > > ! > > aggregate[wan0_in]: dst_host, src_port, dst_port, proto > > aggregate[wan0_out]: src_host, src_port, dst_port, proto > > > > Where pretag2.map: > > set_tag=1 filter='src net 192.168.28.0/24 or src net 192.168.100.0/24' > > set_tag=2 filter='dst net 192.168.28.0/24 or dst net 192.168.100.0/24' > > > > The issue I have with the above config is that no traffic is being > > collected at all. I confirm that when removing the pre_tag filters, > traffic > > is collected, though it is not sorted per direction as I would like to > > have. > > Can I use pre_tag_map and pre_tag_filter with uacctd? I don't see any > > examples for uacctd at > > https://github.com/pmacct/pmacct/blob/master/examples/pretag.map.example > . > > > > Thanx, > > Alex > > > > On Thu, Feb 20, 2020 at 6:33 PM Alex K wrote: > > > > > Hi all, > > > > > > I have a router with multiple interfaces and will need to account > traffic > > > at its several WAN interfaces. My purpose is toaccount the traffic > with the > > > tuple details and the direction. > > > > > > As a test I have compiled the following simple configuration for > pmacctd: > > > > > > ! > > > daemonize: true > > > plugins: print[wan0_in], print[wan0_out] > > > print_refresh_time: 10 > > > print_history: 15m > > > ! > > > print_output[wan0_in]: csv > > > print_output_file[wan0_in]: in_traffic.csv > > > print_output[wan0_out]: csv > > > print_output_file[wan0_out]: out_traffic.csv > > > ! > > > aggregate[wan0_in]: src_host, dst_host, src_port, dst_port, tag > > > aggregate[wan0_out]: src_host, dst_host, src_port, dst_port, tag > > > ! > > > pre_tag_filter[wan0_in]:1 >
Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Hi Alex, Is it possible with the new setup - the one where pre_tag_map does not match anything - the traffic is VLAN-tagged (or MPLS-labelled)? If so, you should adjust filters accordingly and add 'vlan and', ie. "vlan and src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24". Paolo On Fri, Feb 21, 2020 at 01:04:25PM +0200, Alex K wrote: > Working further on this, it seems that for pmacct is sufficient to filter > traffic using only the pre_tag_filter, thus no need for the aggregation > filters. > The issue with this setup though is that I loose the information of the > pre_nat source IP address when monitoring at the WAN interfaces. Due to > this I am switching to uacctd as following: > > ! > daemonize: true > promisc: false > uacctd_group: 1 > !networks_file: networks.lst > !ports_file: ports.lst > ! > pre_tag_map: pretag2.map > pre_tag_filter[print_wan0_in]: 1 > pre_tag_filter[print_wan0_out]: 2 > pre_tag_filter[wan0_in]: 1 > pre_tag_filter[wan0_out]: 2 > ! > plugins: print[print_wan0_in], print[print_wan0_out], mysql[wan0_in], > mysql[wan0_out] > plugin_pipe_size[wan0_in]: 1024000 > plugin_pipe_size[wan0_out]: 1024000 > print_refresh_time: 10 > print_history: 15m > print_output_file_append: true > ! > print_output[print_wan0_in]: csv > print_output_file[print_wan0_in]: in_traffic.csv > print_output[print_wan0_out]: csv > print_output_file[print_wan0_out]: out_traffic.csv > ! > aggregate[print_wan0_in]: dst_host, src_port, dst_port, proto > aggregate[print_wan0_out]: src_host, src_port, dst_port, proto > ! > sql_table[wan0_in]: traffic_wan0_in_%Y%m%d_%H%M > sql_table[wan0_out]: traffic_wan0_out_%Y%m%d_%H%M > ! > sql_table_schema[wan0_in]: traffic_wan0_in.schema > sql_table_schema[wan0_out]: traffic_wan0_out.schema > ! > sql_host: localhost > sql_db : uacct > sql_user : uacct > sql_passwd: uacct > sql_refresh_time: 30 > sql_optimize_clauses: true > sql_history : 24h > sql_history_roundoff: mhd > ! > aggregate[wan0_in]: dst_host, src_port, dst_port, proto > aggregate[wan0_out]: src_host, src_port, dst_port, proto > > Where pretag2.map: > set_tag=1 filter='src net 192.168.28.0/24 or src net 192.168.100.0/24' > set_tag=2 filter='dst net 192.168.28.0/24 or dst net 192.168.100.0/24' > > The issue I have with the above config is that no traffic is being > collected at all. I confirm that when removing the pre_tag filters, traffic > is collected, though it is not sorted per direction as I would like to > have. > Can I use pre_tag_map and pre_tag_filter with uacctd? I don't see any > examples for uacctd at > https://github.com/pmacct/pmacct/blob/master/examples/pretag.map.example. > > Thanx, > Alex > > On Thu, Feb 20, 2020 at 6:33 PM Alex K wrote: > > > Hi all, > > > > I have a router with multiple interfaces and will need to account traffic > > at its several WAN interfaces. My purpose is toaccount the traffic with the > > tuple details and the direction. > > > > As a test I have compiled the following simple configuration for pmacctd: > > > > ! > > daemonize: true > > plugins: print[wan0_in], print[wan0_out] > > print_refresh_time: 10 > > print_history: 15m > > ! > > print_output[wan0_in]: csv > > print_output_file[wan0_in]: in_traffic.csv > > print_output[wan0_out]: csv > > print_output_file[wan0_out]: out_traffic.csv > > ! > > aggregate[wan0_in]: src_host, dst_host, src_port, dst_port, tag > > aggregate[wan0_out]: src_host, dst_host, src_port, dst_port, tag > > ! > > pre_tag_filter[wan0_in]:1 > > pre_tag_filter[wan0_out]:2 > > ! > > pcap_interface: eth0 > > pre_tag_map: pretag.map > > networks_file: networks.lst > > ports_file: ports.lst > > ! > > > > where pretag.map is: > > set_tag=1 filter='ether dst 52:54:00:69:a6:0b' > > set_tag=2 filter='ether src 52:54:00:69:a6:0b' > > > > and networks.lst is: > > 10.100.100.0/24 > > > > It seems that the details output at the CSV are correctly filtered > > according to the tag, thus recording the direction also, based on the MAC > > address of the WAN0 interface. > > > > Is this the correct approach to achieve this or is there any other > > recommended way? Do I need to use aggregate_filters? > > > > Also, although I have set a network filter to capture only 10.100.100.0/24, > > I observe several networks in/out being collected, indicating that the > > network_file directive is ignored or I have misunderstood its purpose. My > > purpose it to collect traffic only generated from subnets that belong to > > configured interfaces of the router. > > > > Thanx for your feedback! > > Alex > > > > > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Pmacct configuration with direction of traffic
Working further on this, it seems that for pmacct is sufficient to filter traffic using only the pre_tag_filter, thus no need for the aggregation filters. The issue with this setup though is that I loose the information of the pre_nat source IP address when monitoring at the WAN interfaces. Due to this I am switching to uacctd as following: ! daemonize: true promisc: false uacctd_group: 1 !networks_file: networks.lst !ports_file: ports.lst ! pre_tag_map: pretag2.map pre_tag_filter[print_wan0_in]: 1 pre_tag_filter[print_wan0_out]: 2 pre_tag_filter[wan0_in]: 1 pre_tag_filter[wan0_out]: 2 ! plugins: print[print_wan0_in], print[print_wan0_out], mysql[wan0_in], mysql[wan0_out] plugin_pipe_size[wan0_in]: 1024000 plugin_pipe_size[wan0_out]: 1024000 print_refresh_time: 10 print_history: 15m print_output_file_append: true ! print_output[print_wan0_in]: csv print_output_file[print_wan0_in]: in_traffic.csv print_output[print_wan0_out]: csv print_output_file[print_wan0_out]: out_traffic.csv ! aggregate[print_wan0_in]: dst_host, src_port, dst_port, proto aggregate[print_wan0_out]: src_host, src_port, dst_port, proto ! sql_table[wan0_in]: traffic_wan0_in_%Y%m%d_%H%M sql_table[wan0_out]: traffic_wan0_out_%Y%m%d_%H%M ! sql_table_schema[wan0_in]: traffic_wan0_in.schema sql_table_schema[wan0_out]: traffic_wan0_out.schema ! sql_host: localhost sql_db : uacct sql_user : uacct sql_passwd: uacct sql_refresh_time: 30 sql_optimize_clauses: true sql_history : 24h sql_history_roundoff: mhd ! aggregate[wan0_in]: dst_host, src_port, dst_port, proto aggregate[wan0_out]: src_host, src_port, dst_port, proto Where pretag2.map: set_tag=1 filter='src net 192.168.28.0/24 or src net 192.168.100.0/24' set_tag=2 filter='dst net 192.168.28.0/24 or dst net 192.168.100.0/24' The issue I have with the above config is that no traffic is being collected at all. I confirm that when removing the pre_tag filters, traffic is collected, though it is not sorted per direction as I would like to have. Can I use pre_tag_map and pre_tag_filter with uacctd? I don't see any examples for uacctd at https://github.com/pmacct/pmacct/blob/master/examples/pretag.map.example. Thanx, Alex On Thu, Feb 20, 2020 at 6:33 PM Alex K wrote: > Hi all, > > I have a router with multiple interfaces and will need to account traffic > at its several WAN interfaces. My purpose is toaccount the traffic with the > tuple details and the direction. > > As a test I have compiled the following simple configuration for pmacctd: > > ! > daemonize: true > plugins: print[wan0_in], print[wan0_out] > print_refresh_time: 10 > print_history: 15m > ! > print_output[wan0_in]: csv > print_output_file[wan0_in]: in_traffic.csv > print_output[wan0_out]: csv > print_output_file[wan0_out]: out_traffic.csv > ! > aggregate[wan0_in]: src_host, dst_host, src_port, dst_port, tag > aggregate[wan0_out]: src_host, dst_host, src_port, dst_port, tag > ! > pre_tag_filter[wan0_in]:1 > pre_tag_filter[wan0_out]:2 > ! > pcap_interface: eth0 > pre_tag_map: pretag.map > networks_file: networks.lst > ports_file: ports.lst > ! > > where pretag.map is: > set_tag=1 filter='ether dst 52:54:00:69:a6:0b' > set_tag=2 filter='ether src 52:54:00:69:a6:0b' > > and networks.lst is: > 10.100.100.0/24 > > It seems that the details output at the CSV are correctly filtered > according to the tag, thus recording the direction also, based on the MAC > address of the WAN0 interface. > > Is this the correct approach to achieve this or is there any other > recommended way? Do I need to use aggregate_filters? > > Also, although I have set a network filter to capture only 10.100.100.0/24, > I observe several networks in/out being collected, indicating that the > network_file directive is ignored or I have misunderstood its purpose. My > purpose it to collect traffic only generated from subnets that belong to > configured interfaces of the router. > > Thanx for your feedback! > Alex > > > ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Pmacct configuration with direction of traffic
Hi all, I have a router with multiple interfaces and will need to account traffic at its several WAN interfaces. My purpose is toaccount the traffic with the tuple details and the direction. As a test I have compiled the following simple configuration for pmacctd: ! daemonize: true plugins: print[wan0_in], print[wan0_out] print_refresh_time: 10 print_history: 15m ! print_output[wan0_in]: csv print_output_file[wan0_in]: in_traffic.csv print_output[wan0_out]: csv print_output_file[wan0_out]: out_traffic.csv ! aggregate[wan0_in]: src_host, dst_host, src_port, dst_port, tag aggregate[wan0_out]: src_host, dst_host, src_port, dst_port, tag ! pre_tag_filter[wan0_in]:1 pre_tag_filter[wan0_out]:2 ! pcap_interface: eth0 pre_tag_map: pretag.map networks_file: networks.lst ports_file: ports.lst ! where pretag.map is: set_tag=1 filter='ether dst 52:54:00:69:a6:0b' set_tag=2 filter='ether src 52:54:00:69:a6:0b' and networks.lst is: 10.100.100.0/24 It seems that the details output at the CSV are correctly filtered according to the tag, thus recording the direction also, based on the MAC address of the WAN0 interface. Is this the correct approach to achieve this or is there any other recommended way? Do I need to use aggregate_filters? Also, although I have set a network filter to capture only 10.100.100.0/24, I observe several networks in/out being collected, indicating that the network_file directive is ignored or I have misunderstood its purpose. My purpose it to collect traffic only generated from subnets that belong to configured interfaces of the router. Thanx for your feedback! Alex ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
