Hi Hendrik,
If your NetFlow/IPIFX exporter implementation is decently done, it may
be all easier than that. There is field type 61 ( see
https://www.iana.org/assignments/ipfix/ipfix.xhtml ) that denotes the
sampling direction. Tipically, ie. in Cisco, it is either 0x00 or absent
if sampling is ingress vs it is always present and set to 0x01 if
sampling is egress.
You can either check if your flows have such a field and are labelled
properly or, more quickly, you can:
1) edit a file /path/to/pretag.map with a one liner that will tag flows
in egress direction (0x01) with a value 100:
tag=100 direction=1
2) complement your current config to read the pretag.map file and filter
out the flows with tag 100 (we actually filter in untagged traffic, that
is, traffic with tag 0):
pre_tag_map: /path/to/pretag.map
pre_tag_filter[foo]: 0
One note on the 'foo' part. That is a plugin name; you can't make
pre_tag_filter a global config directive, it has to be associated to a
specific named plugin. This means, if you are not doing it already, ie.
running only one single unnamed plugin, give it a name. How to do it?
Super simple: you may have a line right now a-la:
plugins: kafka
You should just change it to:
plugins: kafka[foo]
Where 'foo' can be any string of your choice.
Paolo
On 17/5/21 14:40, Hendrik Meyburgh wrote:
Hi.
I have looked at and tested the options over the past few days and
realistically we need to sample both directions at both locations as we
have a different use case we need to satisfy at the network edges in
addition to accounting subscriber traffic. We are investigating with the
routing vendor if there is a way of specifying a sampling interface to
send to a specific collector but we are still waiting for feedback on if
that is possible.
Another option I have been considering is that I should use multiple
pmacct collectors, where the first one filter based on the source ip,
same prefix list for src_host and dst_host, with and the same in_face,
out_face and then tee/replicate that to another collector to recombine
them and to sum_host, I haven't tested that yet, will it work or is
there something else can I try?
Thank you.
On Thu, May 13, 2021 at 2:40 AM Paolo Lucente <[email protected]
<mailto:[email protected]>> wrote:
Hi Hendrik,
What direction are you sampling NetFlow traffic at your edges? Is it
consistent, are you sampling at both place in the same direction,
either
ingress (which would make more sense) or egress (which would make
slight
less sense)? If so, i'd be puzzled why you would get duplicated
traffic;
if, instead, you mix directions or do both at both endpoints, etc.
then,
yeah, that makes sense (and if so we can further analize the scenario).
Paolo
On 12/5/21 11:44, Hendrik Meyburgh wrote:
> Hi.
>
> I have an issue where my setup is causing double counting when using
> sum_host using the below topology. The sampling is set up on the
> interface where the SRC is located and also on the peering edges. My
> config is below, is there something else which I can enable to
stop this
> from happening? We are currently testing setting the same
> observation-domain-id for both routers to see if that will help.
>
> Thank you.
>
> +------------+ +-------------+
> SRC --------- Router1 +<--------->+ Router2
+--------------- DST
> Sampling +--------+---+ +--------+----+.
Sampling
> | |
> | |
> | |
> | |
> | |
> | |
> +--+-----------+--+
> | |
> | pmacct |
> +-----------+
>
> daemonize: true
>
> nfacctd_port: 2100
>
> logfile: /var/log/nfacctd.log
>
> !debug: true
>
> plugins: print[SUM]
>
>
> ! Test2: disable below
>
> nfacctd_renormalize: true
>
> !nfacctd_ext_sampling_rate: 1024
>
> nfacctd_pro_rating: true
>
> !
>
> nfacctd_time_new: true
>
> aggregate[SUM]: sum_host
>
> networks_file[SUM]: /root/pmacct/TARGETS
>
> networks_file_filter[SUM]: true
>
> print_cache_entries[SUM]: 99991
>
> print_refresh_time[SUM]: 300
>
> print_history[SUM]: 5m
>
> print_output[SUM]: csv
>
> print_output_file[SUM]: /root/pmacct/SUM/file-%Y%m%d-%H%M.txt
>
> print_history_roundoff[SUM]: m
>
>
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
<http://www.pmacct.net/#mailinglists>
>
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
