Hello everyone,
I have a backend plugin that uses nfcapd with a bunch of dynamic filters to
generate some rrds that I want (mostly traffic per prefix (input and
output) and top AS (input and output)). So far, it has run for 2-3 years
without major problems, but the volume of data to be searched has increased
and now, the plugin regularly takes over 300s to finish what it has to do
(currently it peaks at 450s).
I am thinking on how to improve performance, and I can tune the some of the
queries (use lists [12 23 34] instead of "or": dst as 12 or dst as 23 or
dst as 34), but I think it will only bring a small speed increase. The real
problem is my code runs nfdump with a set of parameters and then parses the
output. Starting an external process is time (and resource) consuming, but
I guess there's no other way around it.
Instead, I plan on benefiting from my multi-core/high memory system and run
the queries in parallel, by using threads inside the module.
Before I embark in an epic journey of code rewriting, I'd like to ask if I
can expect speed increases from this approach, or if the threads will be
limited to a single cpu/work queue by how the plugin architecture is
implemented.
Do you have other suggestions on how to speed things up?
Thanks,
Adrian
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
