Re: [pmacct-discussion] Looking for value suggestions

[Mark Schouten]

Hi Paolo,

On Mon, Apr 06, 2020 at 01:47:49PM +, Paolo Lucente wrote:
> Do you have any multi-paths between your routers (where you generate
> IPFIX packets) and your collector? Is it possible, in other words, we
> may be looking at an out of order delivery issue? If you are not doing
> that already, can you take a pcap sample on the router (that is, where
> IPFIX data is generated)? Depending if the sequencing issue is notified
> there, we can pin-point (or not!) to out of ordering. If it's not an out
> of order issue it would help a lot if you could send me the pcap trace
> privately for me to inspect it and get further clues.

The machines are 'directly' connected. So no. I made a tcpdump on the
router, and that shows the same issue. I will send you a link to the
dump off-list.

> Wrt buffers: if there is a buffering issue inside pmacct then you are
> notified by the means of warning/error messages so it's easy to spot;
> trickier could be to spot issues between the kernel and pmacct for the
> case of libpcap: you should send a SIGUSR1 to pmacctd ('killall -USR1
> pmacctd' would do) and read output back in the log file. Here it is
> documented what kind of output you should expect (see 'dropped_packets'):

I see no dropped packets.

-- 
Mark Schouten | Tuxis B.V.
KvK: 74698818 | http://www.tuxis.nl/
T: +31 318 200208 | i...@tuxis.nl

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

[pmacct-discussion] Looking for value suggestions

[Mark Schouten]

Hi,

I'm from Tuxis, a small ISP from the Netherlands. We run a small network, with 
three transits and one peering network. Our routers are Debian/Bird-based, and 
perform pretty well, if I may say so myself.

On each router, we run a pmacct per external interface. Primary goal is 
invoicing (through amqp), secondary goal is sFlow and Netflow for obvious 
reasons.

I have sFlow sampling at 1:512 and all is working well there. Netflow however, 
is an issue.

We export Netflow to nprobe, and nprobe complains about Flow Collection Drops. 
And I can confirm that with Wireshark, which reports:
    FlowSequence: 889946358 (expected 888912934)
        [Expert Info (Warning/Sequence): Unexpected flow sequence for domain ID 
77 (expected 888912934, got 889946358)]
            [Unexpected flow sequence for domain ID 77 (expected 888912934, got 
889946358)]
            [Severity level: Warning]
            [Group: Sequence]


So it seems that pmacct is missing data on flows, probably due to my 
configuration. Two questions:

1: How can I see that pmacct buffers or buckets are overrunning?
2: Looking at my current configuration, which values should I alter to achieve 
both (mostly) complete flowdata and not too much CPU usage?


Thanks in advance, also for the cool product that pmacctd is!


daemonize: true
pidfile: /var/run/pmacctd.pid
syslog: daemon

interface: v-amsix

pmacctd_flow_buffer_size: 268435456 !256MB
pmacctd_flow_buffer_buckets: 65536
pmacctd_conntrack_buffer_size: 134217728 !128MB
pmacctd_flow_tcp_lifetime: 3600

plugins: nfprobe[in],nfprobe[out],sfprobe[fnm],amqp
!
plugin_buffer_size: 102400
plugin_pipe_size: 10240

sampling_rate: 1
sampling_rate[fnm]: 512
timestamps_since_epoch: true
geoipv2_file: /var/lib/GeoIP/GeoLite2-Country.mmdb
aggregate: src_host_country, dst_host_country
aggregate: 
src_host,dst_host,src_port,dst_port,proto,src_mac,dst_mac,src_host_country,dst_host_country
amqp_exchange: pmacct
amqp_routing_key: acct
amqp_refresh_time: 300
amqp_history_roundoff: m
amqp_host: 
amqp_user: 
amqp_passwd: 
amqp_exchange_type: direct
amqp_persistent_msg: false
amqp_cache_entries: 262147
amqp_multi_values: 65536
amqp_history: 5m

sfprobe_agentsubid: 1402
sfprobe_receiver: 

nfprobe_receiver: 
nfprobe_version: 10
nfprobe_source_ip: 
nfprobe_direction[in]: in
nfprobe_direction[out]: out
nfprobe_ifindex[in]: 77
nfprobe_ifindex[out]: 77
nfprobe_engine: 77


--
Mark Schouten 

Tuxis, Ede, https://www.tuxis.nl

T: +31 318 200208 
 

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists