[pmacct-discussion] Trafifc level from nfacctd and snmp
Hi,
I have a question - maybe somebody had a similar issue - I'm receiving
netflow from router (Juniper) - they are sampled 1:2000
After the traffic is recalculated by nfacctd - in comparision to statistics
received via snmp - I have strange values - in the lowest traffic level
snmp shows around 550Mbps - in the same time traffic calculated by nfacctd
is ~1.3Gbps - in max point - snmp is showing 6Gbps but nfacctd 3.9 Gbps
I understand that traffic is sampled so it won't be exactly at the same
level as counted by snmp - but isn't it too big difference ?
Instead of this - the characteristics of the traffic is correct - traffic
is growing in the same direction, traffic drops are present in the same
time etc. - only this traffic level..
This is conifguration from router - it's quite simple:
sampling {
input {
rate 2000;
max-packets-per-second 7000;
}
family inet {
output {
flow-server x.x.x.x {
port x;
autonomous-system-type origin;
no-local-dump;
source-address x.x.x.x;
version 5;
}
}
}
}
in nfacctd config file - I recalculate netflows like this:
sql_optimize_clauses: true
sql_dont_try_update: true
sql_multi_values: 1024000
sql_db: pmacct
sql_host: host
sql_passwd: pass
sql_table_version: 7
sql_table_type: bgp
sql_cache_entries: 256000
sql_preprocess: usrf=2000
From this what I checked - the problem - for sure - is not in nfacctd,
netflow data received and recalculated by nfdump was almost the same -
maybe there is something different what I should change/modify to get
the traffic level little more accurate.
Thanks for response
Regards
Adam
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Problem with more than 2 mysql plugins
Hi Paolo, After changing sql_cache_entries from 256k to 64k - updates are made without problem :) Thank You for hint and help. BR Adam 2014-02-06 Adam Bogdan nelr...@gmail.com: Hi Paolo, Thanks for answer - I think the memory is not the problem in this case - I still have some of it available and even swap isn't used but I'll check it with smaller cache_entry value. I had problem with memory but then in logs I get this information: Unable to fork DB writer: Cannot allocate memory - for now I changed few options and the problem is gone. When I changed plugin from mysql to sqlite3 - updates are made. I'll check cache_entries and let u know BR Adam 2014-02-06 Paolo Lucente pa...@pmacct.net: Hi Adam, Is it possible you are running out of memory or so? And maybe as a side result of swapping also CPU is 100%? I see in your config you have 'sql_cache_entries: 256000' which should take quite some memory per each plugin defined. Cheers, Paolo On Thu, Feb 06, 2014 at 02:02:14PM +0100, Adam Bogdan wrote: Hi again, I have a problem with running nfacctd to serve 3 mysql plugins/tables - version nfacctd 1.5.0rc2 config: daemonize: true debug: true pidfile: /var/run/nfacctd_r7.pid syslog: daemon aggregate: tag, src_as, dst_as, peer_src_as, peer_dst_as nfacctd_ip: x.x.x.x nfacctd_port: x nfacctd_time_new: true nfacctd_as_new: fallback nfacctd_net: fallback nfacctd_disable_checks: true networks_file: /etc/pmacct/networks.lst pre_tag_map: /etc/pmacct/pretag.map pre_tag_filter[abc]: 11 pre_tag_filter[ddd]: 10 pre_tag_filter[vcc]: 20 plugins: mysql[abc], mysql[ddd], mysql[vcc] plugin_pipe_size: 6544 plugin_buffer_size: 3 bgp_daemon: true bgp_daemon_ip: x.x.x.x bgp_daemon_max_peers: 10 bgp_peer_src_as_type: bgp bgp_src_as_path_type: bgp bgp_daemon_msglog: false bgp_agent_map: /etc/pmacct/agent.map sql_optimize_clauses: true sql_dont_try_update: true sql_multi_values: 1024000 sql_db: pmacct sql_host: x sql_passwd: x sql_table_version: 7 sql_table_type: bgp sql_cache_entries: 256000 sql_preprocess: usrf=2000 sql_history_roundoff[abc]: m sql_history[abc]: 5m sql_refresh_time[abc]: 300 sql_table[abc]: acct_bgp_abc sql_history_roundoff[ddd]: m sql_history[ddd]: 5m sql_refresh_time[ddd]: 300 sql_table[ddd]: acct_bgp_ddd sql_history_roundoff[vcc]: m sql_history[vcc]: 5m sql_refresh_time[vcc]: 300 sql_table[vcc]: acct_bgp_r7_vcc And this configuration doesn't work :( - If I enable only mysql abc i ddd then it's ok - updates to DB are made and everything is working - when I added vcc - it's dead - nfacctd is running but no updates to DB - when I turn on debugging - I get only keepalives from BGP - neither one update. When I turn off abc and ddd - vcc is working fine (updates etc.) One more hint - when I run config above, I can't kill nfacctd normally - only with option -9 Thanks for help Regards Adam ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Trafifc level from nfacctd and snmp
Hi Adam,
Let me share some thoughts to kick-start the discussion (ie.
inviting people running Juniper to speak up and correct where
needed).
I don't have an explanation why at lowest times renormailzed
NetFlow over-counts SNMP unless sampling rate is somehow not
reported correctly (i have seen this although on C7600: some
line-cards able to report it correctly, others not). If you
are not using sampling_map yet, you could try inserting known
sampling rate values.
For what regards peak hour and NetFlow under-counting compared
to SNMP and the smooth trending of passing from over-counting
to under-counting, i might have a possible explanation: you
are using NetFlow v5 which is centralized, throttled process.
So if 1:2000 reveals too much work for the CPU you are subject
to NetFlow samples not being exported to the collector. This
is solved by using NetFlow v9 and a MS-DPC or in-line IPFIX,
at least on MX series.
Cheers,
Paolo
On Thu, Feb 06, 2014 at 10:45:39AM +0100, Adam Bogdan wrote:
Hi,
I have a question - maybe somebody had a similar issue - I'm receiving
netflow from router (Juniper) - they are sampled 1:2000
After the traffic is recalculated by nfacctd - in comparision to statistics
received via snmp - I have strange values - in the lowest traffic level
snmp shows around 550Mbps - in the same time traffic calculated by nfacctd
is ~1.3Gbps - in max point - snmp is showing 6Gbps but nfacctd 3.9 Gbps
I understand that traffic is sampled so it won't be exactly at the same
level as counted by snmp - but isn't it too big difference ?
Instead of this - the characteristics of the traffic is correct - traffic
is growing in the same direction, traffic drops are present in the same
time etc. - only this traffic level..
This is conifguration from router - it's quite simple:
sampling {
input {
rate 2000;
max-packets-per-second 7000;
}
family inet {
output {
flow-server x.x.x.x {
port x;
autonomous-system-type origin;
no-local-dump;
source-address x.x.x.x;
version 5;
}
}
}
}
in nfacctd config file - I recalculate netflows like this:
sql_optimize_clauses: true
sql_dont_try_update: true
sql_multi_values: 1024000
sql_db: pmacct
sql_host: host
sql_passwd: pass
sql_table_version: 7
sql_table_type: bgp
sql_cache_entries: 256000
sql_preprocess: usrf=2000
From this what I checked - the problem - for sure - is not in nfacctd,
netflow data received and recalculated by nfdump was almost the same -
maybe there is something different what I should change/modify to get
the traffic level little more accurate.
Thanks for response
Regards
Adam
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
