Hi guys, Would be great if someone of the maintainers could comment on my patches, and give some feedback on whether they make sense or propose and alternative/better configuration to achieve a similar setup.
Thanks a lot, Mikhail On Fri, 17 Jan 2020 at 13:02, Mikhail Sennikovsky <mikhail.sennikovs...@cloud.ionos.com> wrote: > > Hi all, > > I had experienced some issues with configuring multiple pcap interfaces with > pmacctd and wanted to clarify them with you and potentially ask for a > better/alternative solution/configuration. > The two patches submitted here actually illustrate the problems I encountered > and the way I had to fix them. > In my setup I have a "firewall entity" which forwards all traffic between two > network interfaces, doing the necessary traffic firewalling/filtering, i.e. > something like: > +------------------------------------------+ > <---> | InterfaceA <->(Firewall)<-> InterfaceB | <---> > +------------------------------------------+ > now I want to generate a NetFlow v9 / IPFIX data for the traffic the Firewall > instance allows in both directions, which means for all egress traffic for > InterfaceA and all egress traffic for InterfaceB. > > So I tried to use pmacctd + its nfprobe plugin for doing that (configs are > listed below).I configured pmacctd with multiple pcap interfaces, specifying > pcap_interfaces_map and the nfprobe plugin (configs are listed below). > The first problem I encountered is that pmacctd does not configure pcap > direction. > I.e. I can not use pcap filter, because I do not have any specific traffic > pattern to determine the egress traffic for both interfaces (i.e. all traffic > is being forwarded between those two interfaces, potentially filtered with > some "firewall rules" in place.) > The pcap direction configuration was explicitly disabled in > 81fe649917036b9ef1ed4b3ea521befcaf36496b, however even before that commit it > apparently did not work, because the pcap_setdirection must be called after > pcap_activate, not before it. > So my first patch actually makes it possible to configure pmacctd to do > pcap_setdirection by introducing a new config variable, pcap_set_direction. > > Now after I ended up with pcap direction working, and configured pmacctd to > listen for the egress traffic on both InterfaceA and InterfaceB, I faced the > second problem, which seems to be a real bug in nfprobe plugin. > nfprobe flow tree does not take interface index into consideration when > searching/aggregating the flow data. This means that for the case multiple > pcap interfaces are being monitored and same src/dst ip/port traffic pattern > is being handled over several of those interfaces, this will all land in the > same FLOW entry. This leads to the issues that flows being handled by one > network interface are actually reported via NetFlow (via Flow InputInt and > OutputInt fields) as being handled by another network interface (held by the > FLOW entry originally created for matching the given src/dst ip/port traffic > pattern). > This perhaps could be worked around by having different nfprobe plugin > instances handling InterfaceA and InterfaceB traffic, but I want to get the > NetFlow data for both interfaces simultaneously, and I might also need to > dynamically add/remove such interface pairs w/o restarting the pmacctd. > The latter could be easily achieved with updating the pcap_interfaces_map > file and sending SIGUSR2 to pmacctd. The same would not be possible however > if I had to add/remove nfprobe plugin instance configurations in pmacctd.conf > So my second patch fixes the above issue by making it possible to configure > nfprobe to take flow interface indexes into consideration when > matching/searching for the FLOW entries in the flow cache tree. The > nfprobe_per_interface_flows config is introduced for that. > > Would be great if someone could have a look into these two patches to see if > they make sense, and/or give some hints on a better/proper way of making the > similar NetFlow configuration. > > Here are my configs for the reference: > pmacctd.conf: > ================ > daemonize: false > pidfile: /var/run/pmacctd.pid > syslog: daemon > > pcap_interfaces_map: /path/to/pcap_interfaces.map > pcap_ifindex: map > ! newly introduced config to tell pmacctd to actually do pcap_setdirection > pcap_set_direction: true > promisc: true > > pmacctd_flow_buffer_buckets: 65536 > pmacctd_flow_buffer_size: 128Mb > > plugins: nfprobe[filtered], print[filtered_p] > plugin_pipe_size: 1048576000 > plugin_buffer_size: 10485760 > > aggregate: src_host, dst_host, src_port, dst_port, in_iface, out_iface > aggregate[filtered]: src_host, dst_host, src_port, dst_port, in_iface, > out_iface > aggregate[filtered_p]: src_host, dst_host, src_port, dst_port, in_iface, > out_iface > > pmacctd_as: file > > refresh_maps: true > pre_tag_map: /path/to/pretag.map > > pre_tag_filter[filtered]: -666 > pre_tag_filter[filtered_p]: -666 > > nfprobe_source_ip: 10.11.12.23/24 > nfprobe_receiver: 10.11.12.15:2055 > nfprobe_version: 9 > nfprobe_timeouts: maxlife=10:general=10:icmp=10:expint=10 > nfprobe_maxflows: 512000 > ! newly introduced config to tell nfprobe plugin to also match flow interface > indexes > ! when matching/searching for the FLOW entries in the flow cache tree > nfprobe_per_interface_flows: true > ================ > > pcap_interfaces.map: > ================ > ifindex=100 ifname=InterfaceA direction=out > ifindex=200 ifname=InterfaceB direction=out > ================ > > pretag.map: > ================ > set_tag=101 > ================ > > Thanks & Regards, > Mikhail > > > Mikhail Sennikovsky (2): > * pmacctd: allow configuring pcap_setdirection > * nfprobe: per-interface flows > > src/cfg.c | 2 ++ > src/cfg.h | 2 ++ > src/cfg_handlers.c | 36 ++++++++++++++++++++++++++++++++++++ > src/cfg_handlers.h | 2 ++ > src/nfprobe_plugin/nfprobe_plugin.c | 8 ++++++++ > src/pmacctd.c | 15 +++++++-------- > 6 files changed, 57 insertions(+), 8 deletions(-) > > -- > 2.7.4 > _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
