Hi Jonathan,
Very good test. That should rule out the possibility i had in mind
this could be data between xx:xx:xx:ff:25:10 and xx:xx:xx:ff:25:11.
You can do another test, to progress this one step further, to see
whether the issue is on the nfprobe plugin or with nfacctd somehow
mis-classifying traffic: in parallel to nfacctd, on the collector
box, capture NetFlow traffic with Wireshark and stop it as soon as
you verify some mis-placed flow: analyzing captured data, helping
yourself also with filtering capabilities of Wireshark, you should
be able to check if the NetFlow export is allright.
If this gives a hint to solve the issue, good. Otherwise i'd have
sufficient information to reproduce the scenario in lab as a next
step.
Cheers,
Paolo
On Fri, Sep 06, 2013 at 06:04:06AM +0000, Jonathan Thorpe wrote:
> Hi Paolo,
>
> Thanks for the reply.
>
> If I do the following with the probe:
>
> 1. Run the pmacctd config with the memory plugin as follows (disabling the
> nfprobe plugin in my original probe config):
>
> plugins: memory[inbound], memory[outbound]
> imt_path[inbound]: /tmp/in.pipe
> imt_path[outbound]: /tmp/out.pipe
>
> 2. Run the following command to see if the ins/outs are mixed shows that
> traffic is collected and separated into in/out as expected.
>
> pmacct -s -p /tmp/in.pipe | awk '{ if ($3 ~ /00:10:db:ff:25:.*/) print }'
> pmacct -s -p /tmp/out.pipe | awk '{ if ($2 ~ /00:10:db:ff:25:.*/) print }'
>
> This just tests to make sure 00:10:db:ff:25:xx does not appear on the inbound
> as a destination address and 00:10:db:ff:25:xx does not appear on the
> outbound as a source address.
>
> The tag IDs are set correctly as per the pre_tag_map, so this leaves the
> nfprobe configuration in the pmacctd probe or the classification in the
> nfacctd (or its associated pre_tag_map).
>
> Thoughts?
>
> Kind Regards,
> Jonathan
>
>
>
>
>
> Hi Jonathan,
>
> Since the four tags in pre_tag_map are mutual exclusive and it
> anyway works first-match-wins fashion, remove the 'return=true'
> part from the equation. Let me know if this solves anything. If
> not, it would be beneficial to know whether it's the probe which
> is mistaken tagging (which is my bet) or the collector mistaken
> classifying then we can take it from there.
>
> Cheers,
> Paolo
>
> On Thu, Sep 05, 2013 at 04:32:37AM +0000, Jonathan Thorpe wrote:
> > Hi All,
> >
> > I have a host with two Ethernet interfaces (eth0 and eth1) which are
> > attached
> > to mirrored ports that I'd like to collect data from and store in a MySQL
> > database for collection with separate in and out tables.
> >
> > There are a large number of VLANs in the mirrored ports and I'm able to
> > work
> > out the direction of the traffic by looking at the source and destination
> > MACs.
> >
> > For example, if the src_mac is xx:xx:xx:ff:25:10 or xx:xx:xx:ff:25:11, I
> > consider the traffic as inbound to the hosts. If the dst_mac is is
> > xx:xx:xx:ff:25:10 or xx:xx:xx:ff:25:11, I consider the traffic to be
> > outbound
> > to the hosts I'm collecting for.
> >
> > My configuration uses two pre_tag_maps - one for the pmacctd process to
> > assign a tag based on the src/dst MAC. Inbound and Outbound traffic each
> > gets
> > a different nfprobe_engine ID as a result.
> >
> > The second pre_tag_map is for the nfacctd process and looks at the engine
> > ID
> > to determine whether the traffic is to be counted on inbound or outbound.
> >
> > This works for the most part, however I'm seeing a few packets in the
> > Inbound
> > table that contain a Destination MAC of or xx:xx:xx:ff:25:11 which should
> > actually be in the Outbound table.
> >
> > Clearly I have misunderstood the how the pre_tag_map and pre_tag_filter
> > works.
> >
> > I've tried this on both 0.14.3 and 1.5rc1 with the same results.
> >
> > My configuration is below.
> >
> > I'd greatly appreciate some assistance in understanding where I've gone
> > wrong.
> >
> > Kind Regards,
> > Jonathan
> >
> >
> >
> >
> >
> > Pmacctd process for eth0
> > ----
> > interface: eth0
> > plugins: nfprobe[inbound], nfprobe[outbound]
> >
> > nfprobe_version: 9
> > nfprobe_receiver: 127.0.0.1:2100
> > nfprobe_source_ip: 127.0.0.1
> >
> > nfprobe_engine[inbound]: 0:12
> > nfprobe_engine[outbound]: 0:13
> >
> > aggregate[inbound]: dst_host, src_host, src_mac, dst_mac, vlan, proto,
> > dst_port, src_port, tag
> > aggregate[outbound]: dst_host, src_host, src_mac, dst_mac, vlan, proto,
> > dst_port, src_port, tag
> >
> > pre_tag_map: /etc/pmacct/pretag.map
> > refresh_maps: true
> > pre_tag_map_entries: 3840
> >
> > pre_tag_filter[inbound]: 100-110
> > pre_tag_filter[outbound]: 200-210
> > ---
> >
> >
> > Pmacctd process for eth1 is exactly the same, except for:
> > ----
> > interface: eth1
> > nfprobe_engine[inbound]: 0:10
> > nfprobe_engine[outbound]: 0:11
> > ----
> >
> > The referenced pretag map is as follows:
> > /etc/pmacct/pretag.map
> > ----
> > ! Inputs
> > id=100 filter='ether src xx:xx:xx:ff:25:10' return=true
> > id=101 filter='ether src xx:xx:xx:ff:25:11' return=true
> >
> > ! Outputs
> > id=200 filter='ether dst xx:xx:xx:ff:25:10' return=true
> > id=201 filter='ether dst xx:xx:xx:ff:25:11' return=true
> > ----
> >
> >
> > Then there's the nfacctd configuration file:
> > ----
> > nfacctd_ip: 127.0.0.1
> > nfacctd_port: 2100
> >
> >
> > nfacctd_time_new: true
> > plugins: mysql[inbound], mysql[outbound]
> >
> > aggregate[inbound]: src_mac, dst_mac, vlan, tag
> > aggregate[outbound]: src_mac, dst_mac, vlan, tag
> >
> > pre_tag_filter[inbound]: 100-110
> > pre_tag_filter[outbound]: 200-210
> >
> > <snipped sql configuration>
> >
> > pre_tag_map: /etc/pmacct/pretag-netflow.map
> >
> > refresh_maps: true
> > pre_tag_map_entries: 3840
> > ----
> >
> > The above referenced /etc/pmacct/pretag-netflow.map is as follows:
> >
> > ----
> > id=100 ip=127.0.0.1 engine_type=0 engine_id=10
> > id=101 ip=127.0.0.1 engine_type=0 engine_id=12
> >
> > id=200 ip=127.0.0.1 engine_type=0 engine_id=11
> > id=201 ip=127.0.0.1 engine_type=0 engine_id=13
> > ----
> >
> > _______________________________________________
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists
>
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
