Hi Seamus, Thanks for the capture from the F5 gear, looking forward to it.
I hope somebody can share their experiences with the frontend tool, i don't personally have any insight into it. Did you already try to contact the developer - not sure he is on the list? Cheers, Paolo On Sun, May 19, 2013 at 11:51:09PM +0000, Seamus Ryan wrote: > Regarding the MySQL performance, this is primarily on query. While MySQL > utilises a significant amount of resources during insert, it is the reading > of data from the "combined" view which takes a very long time. > > (it should be note I am using the DB referenced by > https://github.com/Daniel15/pmacct-frontend/ which has an in/out table and > "combined" view) If the directions are incorrect, please let me know and I > will adjust accordingly. > > Ill try and get you a capture from the F5 gear. > > Cheers, > Seamus > > -----Original Message----- > From: pmacct-discussion [mailto:[email protected]] On > Behalf Of Paolo Lucente > Sent: Friday, May 17, 2013 7:02 PM > To: [email protected] > Subject: Re: [pmacct-discussion] Few questions regarding MySQL, IPFIX and > sFlow > > Hi Seamus, > > About your two issues: > > * 115446 records in a single table are not critical at all, would not > expect poor performances from MySQL even on old/downsized hardware. > Maybe you need to be more specific, ie. where do you verify poor > performance - on insert, on query? Your config is basic but should > work no problems. Only issue i find with it is you use two tables, > ie. acct_v7_in and acct_v7_out, but never do any filtering (ie. no > aggregate_filter or networks_file) resulting in: 1) two tables, > every 5 mins, with exact same content and 2) internal as well as > outer internet IP address endpoints being accounted for traffic > which is probably not what you want. > > * sFlow from F5 gears: i suggest you send me privately a brief capture > (libpcap/tcpdump format, full packet size) of the sFlow traffic so > that i can check what is the issue with it: it should have worked > no problems with the CLI you posted - i remember i had feedback from > people reading sFlow from F5 so not sure where the problem can be. > > Cheers, > Paolo > > On Thu, May 16, 2013 at 01:51:03PM +0000, Seamus Ryan wrote: > > Hi all, > > > > First things first, great project, I am glad I found it! I have been > > hunting around for a decent tool such as this and all of the commercial > > products I have reviewed are horribly bloated or overpriced. > > So I am looking at implementing nfacctd/sfacctd + pmacct-frontend for > > simple flow analysis. Devices I intend on covering are F5's BIG-IP and > > Juniper MX routers (there may be others but this is all that I require at > > present). > > The key issues I am facing are; > > > > > > * MySQL performance - When utilising nfacctd to dump to MySQL from > > our Juniper routers, the performance is very poor. Now this may have > > something to do with the fact that 5 minutes worth of data equalled 115446 > > records (below) so if I need to tune down my config please let me know: > > > > mysql> SELECT count( * ) as total_record FROM acct_v7_in; > > +--------------+ > > | total_record | > > +--------------+ > > | 115446 | > > +--------------+ > > 1 row in set (0.09 sec) > > > > mysql> > > > > > > * sfacctd is not intepreting any data - When I execute "sfacctd -l > > 6343 -P print -r 5" nothing ever shows up in my terminal. So its safe to > > say it will never land in the MySQL DB if I told it to. Strangely when I > > tcpdump like so: tcpdump -i eth0 not port 22 and host x.x.x.x -n on the > > same box (while sfacctd is running) it shows data being accepted from one > > of the devices: > > > > 23:42:11.478367 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:11.478442 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:11.478573 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:11.478648 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:11.478762 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 18, length 1216 > > 23:42:21.488784 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:21.488877 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:21.488988 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:21.489135 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:21.489158 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 18, length 1216 > > 23:42:31.495699 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:31.495738 IP y.y.y.y > x.x.x.x: ICMP y.y.y.y udp port sflow > > unreachable, length 556 > > 23:42:31.495774 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:31.495780 IP y.y.y.y > x.x.x.x: ICMP y.y.y.y udp port sflow > > unreachable, length 556 > > 23:42:31.495887 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:31.495893 IP y.y.y.y > x.x.x.x: ICMP y.y.y.y udp port sflow > > unreachable, length 556 > > 23:42:31.496007 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 24, length 1372 > > 23:42:31.496015 IP y.y.y.y > x.x.x.x: ICMP y.y.y.y udp port sflow > > unreachable, length 556 > > 23:42:31.496033 IP x.x.x.x.58655 > y.y.y.y.sflow: sFlowv5, IPv4 agent > > x.x.x.x, agent-id 18, length 1216 > > 23:42:31.496037 IP y.y.y.y > x.x.x.x: ICMP y.y.y.y udp port sflow > > unreachable, length 556 > > > > You can see roughly half way through, this is when I killed sfacctd so it's > > certainly seeing the data, it's just not doing anything with it. > > > > My nfacctd.conf looks like so (I don't daemonize while testing things out): > > > > ! nfacctd configuration > > daemonize: false > > pidfile: /var/run/nfacctd.pid > > syslog: daemon > > nfacctd_port: 9996 > > > > interface: eth0 > > aggregate: src_host, dst_host > > plugins: mysql[inbound], mysql[outbound] > > sql_table[inbound]: acct_v7_in > > sql_table[outbound]: acct_v7_out > > > > ! storage methods > > sql_host: <removed> > > sql_user: <removed> > > sql_db: <removed> > > sql_passwd: <removed> > > sql_refresh_time: 300 > > sql_history: 5m > > sql_history_roundoff: m > > sql_dont_try_update: true > > sql_table_version: 7 > > > > I would love any assistance getting this to work. Even if only IPFIX I > > would be happy (sFlow can come much later if required). > > > > Cheers, > > Seamus > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
