Hi all,

I had experienced some issues with configuring multiple pcap interfaces with 
pmacctd and wanted to clarify them with you and potentially ask for a 
better/alternative solution/configuration.
The two patches submitted here actually illustrate the problems I encountered 
and the way I had to fix them.
In my setup I have a "firewall entity" which forwards all traffic between two 
network interfaces, doing the necessary traffic firewalling/filtering, i.e. 
something like:
<---> | InterfaceA  <->(Firewall)<->  InterfaceB | <--->
now I want to generate a NetFlow v9 / IPFIX data for the traffic the Firewall 
instance allows in both directions, which means for all egress traffic for 
InterfaceA and all egress traffic for InterfaceB.

So I tried to use pmacctd +  its nfprobe plugin for doing that (configs are 
listed below).I configured pmacctd with multiple pcap interfaces, specifying 
pcap_interfaces_map and the nfprobe plugin (configs are listed below).
The first problem I encountered is that pmacctd does not configure pcap 
I.e. I can not use pcap filter, because I do not have any specific traffic 
pattern to determine the egress traffic for both interfaces (i.e. all traffic 
is being forwarded between those two interfaces, potentially filtered with some 
"firewall rules" in place.)
The pcap direction configuration was explicitly disabled in 
81fe649917036b9ef1ed4b3ea521befcaf36496b, however even before that commit it 
apparently did not work, because the pcap_setdirection must be called after 
pcap_activate, not before it.
So my first patch actually makes it possible to configure pmacctd to do 
pcap_setdirection by introducing a new config variable, pcap_set_direction.

Now after I ended up with pcap direction working, and configured pmacctd to 
listen for the egress traffic on both InterfaceA and InterfaceB, I faced the 
second problem, which seems to be a real bug in nfprobe plugin.
nfprobe flow tree does not take interface index into consideration when 
searching/aggregating the flow data. This means that for the case multiple pcap 
interfaces are being monitored and same src/dst ip/port traffic pattern is 
being handled over several of those interfaces, this will all land in the same 
FLOW entry. This leads to the issues that flows being handled by one network 
interface are actually reported via NetFlow (via Flow InputInt and OutputInt 
fields) as being handled by another network interface (held by the FLOW entry 
originally created for matching the given src/dst ip/port traffic pattern).
This perhaps could be worked around by having different nfprobe plugin 
instances handling InterfaceA and InterfaceB traffic, but I want to get the 
NetFlow data for both interfaces simultaneously, and I might also need to 
dynamically add/remove such interface pairs w/o restarting the pmacctd.
The latter could be easily achieved with updating the pcap_interfaces_map file 
and sending SIGUSR2 to pmacctd. The same would not be possible however if I had 
to add/remove nfprobe plugin instance configurations in pmacctd.conf
So my second patch fixes the above issue by making it possible to configure 
nfprobe to take flow interface indexes into consideration when 
matching/searching for the FLOW entries in the flow cache tree. The 
nfprobe_per_interface_flows config is introduced for that.

Would be great if someone could have a look into these two patches to see if 
they make sense, and/or give some hints on a better/proper way of making the 
similar NetFlow configuration.

Here are my configs for the reference:
daemonize: false
pidfile: /var/run/pmacctd.pid
syslog: daemon

pcap_interfaces_map: /path/to/pcap_interfaces.map
pcap_ifindex: map
! newly introduced config to tell pmacctd to actually do pcap_setdirection
pcap_set_direction: true
promisc: true
pmacctd_flow_buffer_buckets: 65536
pmacctd_flow_buffer_size: 128Mb
plugins: nfprobe[filtered], print[filtered_p]
plugin_pipe_size: 1048576000
plugin_buffer_size: 10485760

aggregate: src_host, dst_host, src_port, dst_port, in_iface, out_iface
aggregate[filtered]: src_host, dst_host, src_port, dst_port, in_iface, out_iface
aggregate[filtered_p]: src_host, dst_host, src_port, dst_port, in_iface, 

pmacctd_as: file

refresh_maps: true
pre_tag_map: /path/to/pretag.map

pre_tag_filter[filtered]: -666
pre_tag_filter[filtered_p]: -666

nfprobe_version: 9
nfprobe_timeouts: maxlife=10:general=10:icmp=10:expint=10
nfprobe_maxflows: 512000
! newly introduced config to tell nfprobe plugin to also match flow interface 
! when matching/searching for the FLOW entries in the flow cache tree
nfprobe_per_interface_flows: true

ifindex=100 ifname=InterfaceA direction=out
ifindex=200 ifname=InterfaceB direction=out


Thanks & Regards,

Mikhail Sennikovsky (2):
  * pmacctd: allow configuring pcap_setdirection
  * nfprobe: per-interface flows

 src/cfg.c                           |  2 ++
 src/cfg.h                           |  2 ++
 src/cfg_handlers.c                  | 36 ++++++++++++++++++++++++++++++++++++
 src/cfg_handlers.h                  |  2 ++
 src/nfprobe_plugin/nfprobe_plugin.c |  8 ++++++++
 src/pmacctd.c                       | 15 +++++++--------
 6 files changed, 57 insertions(+), 8 deletions(-)


pmacct-discussion mailing list

Reply via email to