Re: [Ntop-misc] nprobe scaling...

[Luca Deri]

Hi Scott
thanks for using nProbe. A single instance should be able to collect 10-20k+ 
flows/core, this if you’re able to distribute flows across instances. Export to 
ElasticSearch has been improved (and extended to support the latest version) 
recently. What nProbe version are you using?

In order to assist you I would like you to send
- the exact command line you are using to start nprobe
- how do you balance traffic across the probes running on your system

Thanks Luca

> On 26 Jun 2018, at 07:51, Scott Bossi  wrote:
> 
> We are evaluating nprobe, and the results so far look very good.  We are 
> looking for advise on the best method to scale nprobe.  We have 3 reasonably 
> large linux systems -  32 cpu's/68gb of memory each.  We get about 150k flows 
> per second peak, with an avg of about 60k flows per second.  So far, we have 
> been running many nprobe instances (over 100)  on the same server to scale.   
> Nprobe is using very little cpu or memory, which makes me wonder if there is 
> a better way to scale this, so that one instance can take better advantage of 
> the resources on the server.
> Any advice is appreciated.
>  
>  
> We have also trying to export the data to Elastic, but it appears that the 
> nprobe can’t keep up with the data, as it’s exporting in very small batches, 
> in very small sizes.  Is there a way to fine-tune how the data is exported?
>  
> Thanks. 
>  
> Scott Bossi
> Cyber Threat Operations
> Cyber Operations Engineering
> Raytheon Company
>  
> +1.978.436.3750 business
> scott.v.bo...@raytheon.com 
>  
>  
> 880 Technology Park Drive
> Billerica, MA 01821-4164 USA
> www.raytheon.com 
>  
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it 
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc 
> 
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] nprobe scaling...

[Scott Bossi]

We are evaluating nprobe, and the results so far look very good.  We are 
looking for advise on the best method to scale nprobe.  We have 3 reasonably 
large linux systems -  32 cpu's/68gb of memory each.  We get about 150k flows 
per second peak, with an avg of about 60k flows per second.  So far, we have 
been running many nprobe instances (over 100)  on the same server to scale.   
Nprobe is using very little cpu or memory, which makes me wonder if there is a 
better way to scale this, so that one instance can take better advantage of the 
resources on the server.

Any advice is appreciated.


We have also trying to export the data to Elastic, but it appears that the 
nprobe can't keep up with the data, as it's exporting in very small batches, in 
very small sizes.  Is there a way to fine-tune how the data is exported?

Thanks.

Scott Bossi
Cyber Threat Operations
Cyber Operations Engineering
Raytheon Company

+1.978.436.3750 business
scott.v.bo...@raytheon.com


880 Technology Park Drive
Billerica, MA 01821-4164 USA
www.raytheon.com

___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc