Hi Slava,
Although tagging can play in your case a key (negative) role under
sustained loads, i wouldn't know if it is the prime contributor to
such hang ups.
The log below tells that either the router itself is unable to
export all the NetFlow data or such data gets lost before making
it to the collector (network, kernel buffers, etc.). Such sequence
checks can be avoided with the aim of avoid massive logging and in
turn relief CPU load: nfacctd_disable_checks set to true.
What occurs when the Core Process has not time to handle all traffic?
Well, nfacctd reads data from a socket; and a socket at the very end
manages a buffer of a certain size. If nfacctd is too slow to pick
data out of the buffer compared to the arrival rate, there will be
some data loss. At this propo: is buffering enabled within nfacctd
(ie. plugin_pipe_size, plugin_buffer_size) ?
Is it not also an idea, if possible (depends on the router) and for
the benefit of the whole solution, to introduce sampled NetFlow?
Cheers,
Paolo
On Mon, Oct 26, 2009 at 07:57:17PM +0200, Slava Dubrovskiy wrote:
[ ... ]
Throughout our conversation about traffic accounting has noticed that
periodically the daemon nfacctd hangs up.
It happens when the quantity of packages strongly increases more then
50kpps (during DDoS).
In log I see:
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '4333297' but
received '403' collector=???^^B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '405' but
received '406' collector=???^^B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '407' but
received '420' collector=^H^_B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '421' but
received '432' collector=^T^_B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '433' but
received '446' collector=^_B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '447' but
received '456' collector=,^_B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '457' but
received '463' collector=3^_B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '464' but
received '478' collector=B^_B:8818 agent=192.168.21.1:129
Oct 25 18:59:48 stat nfacctd[3379]: WARN: expecting flow '479' but
received '4333400' collector=X^_B:8818 agent=192.168.21.1:129
After this nfacct stop listen port and not working.
Question:
What occurs, when the Core Process has not time to handle all traffic?
How it is possible to increase productivity Pre-Tagging?
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
