Hi Raphael, We are doing one measurment for each netflow exporter. Also we keep only the data for 60 days ( retention policy on influxdb ). On top of that we run continuous queries to generate a new measurment for total traffic in/ total traffic in from transit/peers , this are the most used graphs by our noc. Even with above influxdb still uses ~10G of ram constant. I beleive this can be optimised if you don't use that many primitives and/or use less tags.
Ram is cheap anyway so for us it does the job. Regards, Catalin On Thu, Nov 10, 2016 at 8:43 PM, Raphael Mazelier <r...@futomaki.net> wrote: > > > On 10/11/2016 20:58, Catalin Petrescu wrote: > >> Hi Robert, >> >> We had good results with grafana and influxdb using below : >> -src_as,dst_as,peer_src_as,peer_dst_as,peer_src_ip,as_path, >> dst_net,src_net,dst_mask,src_mask,in_iface,std_comm,med,proto,src_as_path >> >> We export the flow records from pmacctd to kafka then import into influxdb >> and from there is just a matter of creating the templates. The only >> custom >> thing is kafka to influx , and we use kafka-influxdb with a inhouse >> parser * >> (kafka-influxdb). <https://github.com/mre/kafka-influxdb>* take a look >> >> * here >> <http://cpmarvin.blogspot.co.uk/2016/08/open-source-netflow- >> analysis-tool.html>* >> >> >> > Very interesting. I've made something similar using custom amqp consumer. > Is influxdb support well as much data ? because you basically insert one ts > point for one flow entry. > > -- > Raphael Mazelier > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists >
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists