Thank you Paolo,

I see I can use aggregation filters also. So I guess will find a way to
implement what is needed without having a convoluted configuration file.

cheers,
Alex

On Thu, Feb 27, 2020 at 12:24 PM Paolo Lucente <pa...@pmacct.net> wrote:

>
> Hi Alex,
>
> Ack. The other way you could "filter" out is with a networks_file: in
> there you specify the network(s) you are interested in following the
> example here:
>
> https://github.com/pmacct/pmacct/blob/master/examples/networks.lst.example
>
> In the simplest case, you just want to list networks of interest one per
> line. Then in the config you want to set 'networks_file_filter: true' as
> well. This is kind-of filtering: networks / IPs not of interest will be
> just zeroed out and rolled up as a 0.0.0.0 src_host / dst_host.
>

> Paolo
>
> On Wed, Feb 26, 2020 at 11:32:31AM +0200, Alex K wrote:
> > Hi Paolo,
> >
> > On Tue, Feb 25, 2020 at 6:41 PM Paolo Lucente <pa...@pmacct.net> wrote:
> >
> > >
> > > Hi Alex,
> > >
> > > Thanks for your feedback. I see you did run "tcpdump -n -vv -i nflog:1"
> > > which is equivalent to run uacctd without any filters; as you may know,
> > > you can append a BPF-style filter to the tcpdump command-line,
> precisely
> > > as you express it in pre_tag_map. Can you give that a try and see if
> you
> > > get any luck?
> > >
> > Bad luck... I get:
> > tcpdump -nvv -i  nflog:1 src net 192.168.28.0/24
> > tcpdump: NFLOG link-layer type filtering not implemented
> > It seems that filtering at nflog interface is not supported.
> > Running tcpdump -nvv -i eth0 src net 192.168.28.0/24 does capture
> traffic
> > normally.
> > Is there any other way I could apply some filtering with uacctd? I need
> to
> > use uacctd since I get all the pre-nat, post-nat details of the flows, so
> > as to account traffic at the WAN interfaces with the real source details.
> >
> >
> > > My expextation is: if something does not work with pre_tag_map, it
> > > should also not work with tcpdump; if you work out a filter to work
> > > against tcpdump, that should work in pre_tag_map as well. Any
> disconnect
> > > among the two may bring the scent of a bug.
> > >
> > > Paolo
> > >
> > > On Tue, Feb 25, 2020 at 11:20:21AM +0200, Alex K wrote:
> > > > Here is the output when running in debug mode:
> > > >
> > > > INFO ( default/core ): Linux NetFilter NFLOG Accounting Daemon,
> uacctd
> > > > (20200222-01)
> > > > INFO ( default/core ):  '--prefix=/usr' '--enable-mysql'
> '--enable-nflog'
> > > > '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins'
> > > > '--enable-bmp-bins' '--enable-st-bins'
> > > > INFO ( default/core ): Reading configuration file
> > > > '/root/pmacct/uacctd2.conf'.
> > > > INFO ( print_wan0_in/print ): plugin_pipe_size=4096000 bytes
> > > > plugin_buffer_size=280 bytes
> > > > INFO ( print_wan0_in/print ): ctrl channel: obtained=212992 bytes
> > > > target=117024 bytes
> > > > INFO ( print_wan0_out/print ): plugin_pipe_size=4096000 bytes
> > > > plugin_buffer_size=280 bytes
> > > > INFO ( print_wan0_out/print ): ctrl channel: obtained=212992 bytes
> > > > target=117024 bytes
> > > > INFO ( print_wan0_in/print ): cache entries=16411 base cache
> > > > memory=54878384 bytes
> > > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > > INFO ( print_wan0_out/print ): cache entries=16411 base cache
> > > > memory=54878384 bytes
> > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > > INFO ( default/core ): [pretag2.map] (re)loading map.
> > > > INFO ( default/core ): [pretag2.map] map successfully (re)loaded.
> > > > INFO ( default/core ): Successfully connected Netlink NFLOG socket
> > > >
> > > > It doesn't seem to have any issues loading the maps, though it is not
> > > > collecting anything. When capturing with tcpdump I see packets going
> > > > through:
> > > >
> > > > tcpdump -n -vv -i nflog:1
> > > > 09:16:05.831131 IP (tos 0x0, ttl 64, id 36511, offset 0, flags [DF],
> > > proto
> > > > ICMP (1), length 84)
> > > >     192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 1,
> length
> > > 64
> > > > 09:16:05.831362 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none],
> proto
> > > > ICMP (1), length 84)
> > > >     8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 1,
> length 64
> > > > 09:16:05.831392 IP (tos 0x0, ttl 64, id 36682, offset 0, flags [DF],
> > > proto
> > > > ICMP (1), length 84)
> > > >     192.168.28.11 > 8.8.8.8: ICMP echo request, id 17353, seq 2,
> length
> > > 64
> > > > 09:16:06.855200 IP (tos 0x0, ttl 49, id 0, offset 0, flags [none],
> proto
> > > > ICMP (1), length 84)
> > > >     8.8.8.8 > 192.168.28.11: ICMP echo reply, id 17353, seq 2,
> length 64
> > > >
> > > > The pmacct  version I am running is latest master.
> > > > Thank you for your assistance.
> > > >
> > > > Alex
> > > >
> > > >
> > > > On Mon, Feb 24, 2020 at 6:20 PM Alex K <rightkickt...@gmail.com>
> wrote:
> > > >
> > > > > Hi Paolo,
> > > > >
> > > > > On Sat, Feb 22, 2020 at 4:18 PM Paolo Lucente <pa...@pmacct.net>
> > > wrote:
> > > > >
> > > > >>
> > > > >> Hi Alex,
> > > > >>
> > > > >> Is it possible with the new setup - the one where pre_tag_map
> does not
> > > > >> match anything - the traffic is VLAN-tagged (or MPLS-labelled)?
> If so,
> > > > >> you should adjust filters accordingly and add 'vlan and', ie.
> "vlan
> > > and
> > > > >> src net 192.168.28.0/24 or vlan and src net 192.168.100.0/24".
> > > > >>
> > > > > The traffic is not VLAN or MPLS. It is simple one. I confirm I can
> > > collect
> > > > > traffic when removing the pretag directives. Also when stopping
> > > uacctd, I
> > > > > can capture traffic at nflog:1 interface.
> > > > > I simplified the configuration as below:
> > > > >
> > > > > !
> > > > > daemonize: true
> > > > > promisc:   false
> > > > > uacctd_group: 1
> > > > > !
> > > > > pre_tag_map: pretag2.map
> > > > > pre_tag_filter[print_wan0_in]: 1
> > > > > pre_tag_filter[print_wan0_out]: 2
> > > > > !
> > > > > !-------------------------------------------------
> > > > > plugins: print[print_wan0_in], print[print_wan0_out]
> > > > > print_refresh_time: 10
> > > > > print_history: 15m
> > > > > print_output_file_append: true
> > > > > !
> > > > > print_output[print_wan0_in]: csv
> > > > > print_output[print_wan0_out]: csv
> > > > > print_output_file[print_wan0_in]: traffic-wan0-in.csv
> > > > > print_output_file[print_wan0_out]: traffic-wan0-out.csv
> > > > > !
> > > > > aggregate[print_wan0_in]: tag, src_host, dst_host, src_port,
> dst_port,
> > > > > proto
> > > > > aggregate[print_wan0_out]: tag, src_host, dst_host, src_port,
> dst_port,
> > > > > proto
> > > > > !
> > > > >
> > > > > with pretag2.map
> > > > > set_tag=1 filter='src net 192.168.28.0/24'
> > > > > set_tag=2 filter='dst net 192.168.28.0/24'
> > > > >
> > > > > As soon as I enable the pretag directives as below, I do not see
> any
> > > > > traffic being collected from uacctd at NFLOG goup 1
> > > > >
> > > > > pre_tag_map: pretag2.map
> > > > > pre_tag_filter[print_wan0_in]: 1
> > > > > pre_tag_filter[print_wan0_out]: 2
> > > > >
> > > > > I am running pmacct 1.7.4.
> > > > >
> > > > >
> > > > >> Paolo
> > > > >>
> > > > >> On Fri, Feb 21, 2020 at 01:04:25PM +0200, Alex K wrote:
> > > > >> > Working further on this, it seems that for pmacct is sufficient
> to
> > > > >> filter
> > > > >> > traffic using only the pre_tag_filter, thus no need for the
> > > aggregation
> > > > >> > filters.
> > > > >> > The issue with this setup though is that I loose the
> information of
> > > the
> > > > >> > pre_nat source IP address when monitoring at the WAN interfaces.
> > > Due to
> > > > >> > this I am switching to uacctd as following:
> > > > >> >
> > > > >> > !
> > > > >> > daemonize: true
> > > > >> > promisc:   false
> > > > >> > uacctd_group: 1
> > > > >> > !networks_file: networks.lst
> > > > >> > !ports_file: ports.lst
> > > > >> > !
> > > > >> > pre_tag_map: pretag2.map
> > > > >> > pre_tag_filter[print_wan0_in]: 1
> > > > >> > pre_tag_filter[print_wan0_out]: 2
> > > > >> > pre_tag_filter[wan0_in]: 1
> > > > >> > pre_tag_filter[wan0_out]: 2
> > > > >> > !
> > > > >> > plugins: print[print_wan0_in], print[print_wan0_out],
> > > mysql[wan0_in],
> > > > >> > mysql[wan0_out]
> > > > >> > plugin_pipe_size[wan0_in]: 1024000
> > > > >> > plugin_pipe_size[wan0_out]: 1024000
> > > > >> > print_refresh_time: 10
> > > > >> > print_history: 15m
> > > > >> > print_output_file_append: true
> > > > >> > !
> > > > >> > print_output[print_wan0_in]: csv
> > > > >> > print_output_file[print_wan0_in]: in_traffic.csv
> > > > >> > print_output[print_wan0_out]: csv
> > > > >> > print_output_file[print_wan0_out]: out_traffic.csv
> > > > >> > !
> > > > >> > aggregate[print_wan0_in]: dst_host, src_port, dst_port, proto
> > > > >> > aggregate[print_wan0_out]: src_host, src_port, dst_port, proto
> > > > >> > !
> > > > >> > sql_table[wan0_in]: traffic_wan0_in_%Y%m%d_%H%M
> > > > >> > sql_table[wan0_out]: traffic_wan0_out_%Y%m%d_%H%M
> > > > >> > !
> > > > >> > sql_table_schema[wan0_in]: traffic_wan0_in.schema
> > > > >> > sql_table_schema[wan0_out]: traffic_wan0_out.schema
> > > > >> > !
> > > > >> > sql_host: localhost
> > > > >> > sql_db : uacct
> > > > >> > sql_user : uacct
> > > > >> > sql_passwd: uacct
> > > > >> > sql_refresh_time: 30
> > > > >> > sql_optimize_clauses: true
> > > > >> > sql_history : 24h
> > > > >> > sql_history_roundoff: mhd
> > > > >> > !
> > > > >> > aggregate[wan0_in]: dst_host, src_port, dst_port, proto
> > > > >> > aggregate[wan0_out]: src_host, src_port, dst_port, proto
> > > > >> >
> > > > >> > Where pretag2.map:
> > > > >> > set_tag=1 filter='src net 192.168.28.0/24 or src net
> > > 192.168.100.0/24'
> > > > >> > set_tag=2 filter='dst net 192.168.28.0/24 or dst net
> > > 192.168.100.0/24'
> > > > >> >
> > > > >> > The issue I have with the above config is that no traffic is
> being
> > > > >> > collected at all. I confirm that when removing the pre_tag
> filters,
> > > > >> traffic
> > > > >> > is collected, though it is not sorted per direction as I would
> like
> > > to
> > > > >> > have.
> > > > >> > Can I use pre_tag_map and pre_tag_filter with uacctd? I don't
> see
> > > any
> > > > >> > examples for uacctd at
> > > > >> >
> > > > >>
> > >
> https://github.com/pmacct/pmacct/blob/master/examples/pretag.map.example.
> > > > >> >
> > > > >> > Thanx,
> > > > >> > Alex
> > > > >> >
> > > > >> > On Thu, Feb 20, 2020 at 6:33 PM Alex K <rightkickt...@gmail.com
> >
> > > wrote:
> > > > >> >
> > > > >> > > Hi all,
> > > > >> > >
> > > > >> > > I have a router with multiple interfaces and will need to
> account
> > > > >> traffic
> > > > >> > > at its several WAN interfaces. My purpose is toaccount the
> traffic
> > > > >> with the
> > > > >> > > tuple details and the direction.
> > > > >> > >
> > > > >> > > As a test I have compiled the following simple configuration
> for
> > > > >> pmacctd:
> > > > >> > >
> > > > >> > > !
> > > > >> > > daemonize: true
> > > > >> > > plugins: print[wan0_in], print[wan0_out]
> > > > >> > > print_refresh_time: 10
> > > > >> > > print_history: 15m
> > > > >> > > !
> > > > >> > > print_output[wan0_in]: csv
> > > > >> > > print_output_file[wan0_in]: in_traffic.csv
> > > > >> > > print_output[wan0_out]: csv
> > > > >> > > print_output_file[wan0_out]: out_traffic.csv
> > > > >> > > !
> > > > >> > > aggregate[wan0_in]: src_host, dst_host, src_port, dst_port,
> tag
> > > > >> > > aggregate[wan0_out]: src_host, dst_host, src_port, dst_port,
> tag
> > > > >> > > !
> > > > >> > > pre_tag_filter[wan0_in]:1
> > > > >> > > pre_tag_filter[wan0_out]:2
> > > > >> > > !
> > > > >> > > pcap_interface: eth0
> > > > >> > > pre_tag_map: pretag.map
> > > > >> > > networks_file: networks.lst
> > > > >> > > ports_file: ports.lst
> > > > >> > > !
> > > > >> > >
> > > > >> > > where pretag.map is:
> > > > >> > > set_tag=1 filter='ether dst 52:54:00:69:a6:0b'
> > > > >> > > set_tag=2 filter='ether src 52:54:00:69:a6:0b'
> > > > >> > >
> > > > >> > > and networks.lst is:
> > > > >> > > 10.100.100.0/24
> > > > >> > >
> > > > >> > > It seems that the details output at the CSV are correctly
> filtered
> > > > >> > > according to the tag, thus recording the direction also,
> based on
> > > the
> > > > >> MAC
> > > > >> > > address of the WAN0 interface.
> > > > >> > >
> > > > >> > > Is this the correct approach to achieve this or is there any
> other
> > > > >> > > recommended way? Do I need to use aggregate_filters?
> > > > >> > >
> > > > >> > > Also, although I have set a network filter to capture only
> > > > >> 10.100.100.0/24,
> > > > >> > > I observe several networks in/out being collected, indicating
> > > that the
> > > > >> > > network_file directive is ignored or I have misunderstood its
> > > > >> purpose. My
> > > > >> > > purpose it to collect traffic only generated from subnets that
> > > belong
> > > > >> to
> > > > >> > > configured interfaces of the router.
> > > > >> > >
> > > > >> > > Thanx for your feedback!
> > > > >> > > Alex
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >>
> > > > >> > _______________________________________________
> > > > >> > pmacct-discussion mailing list
> > > > >> > http://www.pmacct.net/#mailinglists
> > > > >>
> > > > >>
> > > > >> _______________________________________________
> > > > >> pmacct-discussion mailing list
> > > > >> http://www.pmacct.net/#mailinglists
> > > > >>
> > > > >
> > >
>
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Reply via email to