Re: [pmacct-discussion] Pmacct configuration with direction of traffic

2020-02-27 Thread Alex K
Thank you Paolo,

I see I can use aggregation filters also. So I guess will find a way to
implement what is needed without having a convoluted configuration file.

cheers,
Alex

On Thu, Feb 27, 2020 at 12:24 PM Paolo Lucente  wrote:

>
> Hi Alex,
>
> Ack. The other way you could "filter" out is with a networks_file: in
> there you specify the network(s) you are interested in following the
> example here:
>
> https://github.com/pmacct/pmacct/blob/master/examples/networks.lst.example
>
> In the simplest case, you just want to list networks of interest one per
> line. Then in the config you want to set 'networks_file_filter: true' as
> well. This is kind-of filtering: networks / IPs not of interest will be
> just zeroed out and rolled up as a 0.0.0.0 src_host / dst_host.
>

> Paolo
>
> On Wed, Feb 26, 2020 at 11:32:31AM +0200, Alex K wrote:
> > Hi Paolo,
> >
> > On Tue, Feb 25, 2020 at 6:41 PM Paolo Lucente  wrote:
> >
> > >
> > > Hi Alex,
> > >
> > > Thanks for your feedback. I see you did run "tcpdump -n -vv -i nflog:1"
> > > which is equivalent to run uacctd without any filters; as you may know,
> > > you can append a BPF-style filter to the tcpdump command-line,
> precisely
> > > as you express it in pre_tag_map. Can you give that a try and see if
> you
> > > get any luck?
> > >
> > Bad luck... I get:
> > tcpdump -nvv -i  nflog:1 src net 192.168.28.0/24
> > tcpdump: NFLOG link-layer type filtering not implemented
> > It seems that filtering at nflog interface is not supported.
> > Running tcpdump -nvv -i eth0 src net 192.168.28.0/24 does capture
> traffic
> > normally.
> > Is there any other way I could apply some filtering with uacctd? I need
> to
> > use uacctd since I get all the pre-nat, post-nat details of the flows, so
> > as to account traffic at the WAN interfaces with the real source details.
> >
> >
> > > My expextation is: if something does not work with pre_tag_map, it
> > > should also not work with tcpdump; if you work out a filter to work
> > > against tcpdump, that should work in pre_tag_map as well. Any
> disconnect
> > > among the two may bring the scent of a bug.
> > >
> > > Paolo
> > >
> > > On Tue, Feb 25, 2020 at 11:20:21AM +0200, Alex K wrote:
> > > > Here is the output when running in debug mode:
> > > >
> > > > INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon,
> uacctd
> > > > (20200222-01)
> > > > INFO ( default/core ):  '--prefix=/usr' '--enable-mysql'
> '--enable-nflog'
> > > > '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins'
> > > > '--enable-bmp-bins' '--enable-st-bins'
> > > > INFO ( default/core ): Reading configuration file
> > > > '/root/pmacct/uacctd2.conf'.
> > > > INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes
> > > > plugin_buffer_size=280 bytes
> > > > INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes
> > > > target=117024 bytes
> > > > INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes
> > > > plugin_buffer_size=280 bytes
> > > > INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes
> > > > target=117024 bytes
> > > > INFO ( print_wan0_in/print ): cache entries=16411 base cache
> > > > memory=54878384 bytes
> > > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > > INFO ( print_wan0_out/print ): cache entries=16411 base cache
> > > > memory=54878384 bytes
> > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > > INFO ( default/core ): Successfully connected Netlink NFLOG socket
> > > >
> > > > It doesn't seem to have any issues loading the maps, though it is not
> > > > collecting anything. When capturing with tcpdump I see packets going
> > > > through:
> > > >
> > > > tcpdump -n -vv -i nflog:1
> > > > 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF],
> > > proto
> > > > ICMP (1), length 84)
> > > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1,
> length
> > > 64
> > > > 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none],
> proto
> > > > ICMP (1), length 84)
> > > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1,
> length 64
> > > > 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF],
> > > proto
> > > > ICMP (1), length 84)
> > > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2,
> length
> > > 64
> > > > 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none],
> proto
> > > > ICMP (1), length 84)
> > > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2,
> length 64
> > > >
> > > > The pmacct  version I am running is latest master.
> > > > Thank you for your assistance.
> > > >
> > > > Alex
> > > >
> > > >
> > > > On Mon, Feb 24, 2020 at 6:20 PM Alex K 
> wrote:
> > 

Re: [pmacct-discussion] Pmacct configuration with direction of traffic

2020-02-27 Thread Paolo Lucente


Hi Alex,

Ack. The other way you could "filter" out is with a networks_file: in
there you specify the network(s) you are interested in following the
example here:

https://github.com/pmacct/pmacct/blob/master/examples/networks.lst.example

In the simplest case, you just want to list networks of interest one per
line. Then in the config you want to set 'networks_file_filter: true' as
well. This is kind-of filtering: networks / IPs not of interest will be
just zeroed out and rolled up as a 0.0.0.0 src_host / dst_host.

Paolo

On Wed, Feb 26, 2020 at 11:32:31AM +0200, Alex K wrote:
> Hi Paolo,
> 
> On Tue, Feb 25, 2020 at 6:41 PM Paolo Lucente  wrote:
> 
> >
> > Hi Alex,
> >
> > Thanks for your feedback. I see you did run "tcpdump -n -vv -i nflog:1"
> > which is equivalent to run uacctd without any filters; as you may know,
> > you can append a BPF-style filter to the tcpdump command-line, precisely
> > as you express it in pre_tag_map. Can you give that a try and see if you
> > get any luck?
> >
> Bad luck... I get:
> tcpdump -nvv -i  nflog:1 src net 192.168.28.0/24
> tcpdump: NFLOG link-layer type filtering not implemented
> It seems that filtering at nflog interface is not supported.
> Running tcpdump -nvv -i eth0 src net 192.168.28.0/24 does capture traffic
> normally.
> Is there any other way I could apply some filtering with uacctd? I need to
> use uacctd since I get all the pre-nat, post-nat details of the flows, so
> as to account traffic at the WAN interfaces with the real source details.
> 
> 
> > My expextation is: if something does not work with pre_tag_map, it
> > should also not work with tcpdump; if you work out a filter to work
> > against tcpdump, that should work in pre_tag_map as well. Any disconnect
> > among the two may bring the scent of a bug.
> >
> > Paolo
> >
> > On Tue, Feb 25, 2020 at 11:20:21AM +0200, Alex K wrote:
> > > Here is the output when running in debug mode:
> > >
> > > INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon, uacctd
> > > (20200222-01)
> > > INFO ( default/core ):  '--prefix=/usr' '--enable-mysql' '--enable-nflog'
> > > '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins'
> > > '--enable-bmp-bins' '--enable-st-bins'
> > > INFO ( default/core ): Reading configuration file
> > > '/root/pmacct/uacctd2.conf'.
> > > INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes
> > > plugin_buffer_size=280 bytes
> > > INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes
> > > target=117024 bytes
> > > INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes
> > > plugin_buffer_size=280 bytes
> > > INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes
> > > target=117024 bytes
> > > INFO ( print_wan0_in/print ): cache entries=16411 base cache
> > > memory=54878384 bytes
> > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > INFO ( print_wan0_out/print ): cache entries=16411 base cache
> > > memory=54878384 bytes
> > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > INFO ( default/core ): Successfully connected Netlink NFLOG socket
> > >
> > > It doesn't seem to have any issues loading the maps, though it is not
> > > collecting anything. When capturing with tcpdump I see packets going
> > > through:
> > >
> > > tcpdump -n -vv -i nflog:1
> > > 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF],
> > proto
> > > ICMP (1), length 84)
> > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1, length
> > 64
> > > 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto
> > > ICMP (1), length 84)
> > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1, length 64
> > > 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF],
> > proto
> > > ICMP (1), length 84)
> > > 192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2, length
> > 64
> > > 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none], proto
> > > ICMP (1), length 84)
> > > 8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2, length 64
> > >
> > > The pmacct  version I am running is latest master.
> > > Thank you for your assistance.
> > >
> > > Alex
> > >
> > >
> > > On Mon, Feb 24, 2020 at 6:20 PM Alex K  wrote:
> > >
> > > > Hi Paolo,
> > > >
> > > > On Sat, Feb 22, 2020 at 4:18 PM Paolo Lucente 
> > wrote:
> > > >
> > > >>
> > > >> Hi Alex,
> > > >>
> > > >> Is it possible with the new setup - the one where pre_tag_map does not
> > > >> match anything - the traffic is VLAN-tagged (or MPLS-labelled)? If so,
> > > >> you should adjust filters accordingly and add 'vlan and', ie. "vlan
> > and
> > > >> src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24".
> > > >>
>