To an extent it very much depends on the use case. I have seen over a million
EPS on a six node cluster for pcap and basic net flow. If you add a lot of
complex enrichment and profiling that will obviously increase the load. Tuning
the components for the workload can also make a significant difference. There
are some good tips on that in the tuning guide in the source.
It would be great to hear some of the experiences other people on the list have
had on eps and infrastructure for deployments. If anyone can post specs of a
deployments, that would be fantastic to see.
Simon
> On 17 Oct 2017, at 20:14, ed d wrote:
>
> Is there a rough guide to match EPS to an architectural sizing guide? I know
> its very difficult to extrapolate out, but a rough estimate would be nice.
> This may have already been attempted, and if yes, then please disregard.
>
>
> Or can anyone share what they have found to work best?
>
>
> For example,
>
> POC - 1 machine
> 1 big machine (16 CPU, 128 RAM, 5 Tb HDD)
>
> 100 EPS - 3 machines
> 1 Nifi (8 CPU, 64 RAM, 5 Tb HDD)
> 1 Hadoop/Metron (8 CPU, 64 RAM, 5 Tb HDD)
> 1 Elasticsearch/Kibana (8 CPU, 64 RAM, 5 Tb HDD)
>
> 1000 EPS - 8 machines
> 2 Nifi cliustered (8 CPU, 64 RAM, 5 Tb HDD)
> 2 Hadoop (16 CPU, 128 RAM, 20 Tb HDD)
> 1 Metron (16 CPU, 128 RAM, 1 Tb HDD)
> 1 Elasticsearch data (8 CPU, 64 RAM, 20 Tb HDD)
> 1 Elasticsearch master (8 CPU, 64 RAM, 1 Tb HDD)
> 1 Kibana (8 CPU, 64 RAM, 1 Tb HDD)
>
> 1 EPS - 14 machines
> 4 Nifi clustered (16 CPU, 64 RAM, 5 Tb HDD)
> 2 Hadoop (32 CPU, 128 RAM, 5 Tb HDD)
> 2 Hadoop Data Nodes (32 CPU, 128 RAM, 40 Tb HDD)
> 1 Metron (16 CPU, 128 RAM, 5 Tb HDD)
> 1 Zeppelin (32 CPU, 128 RAM, 5 Tb HDD)
> 2 ES data (32 CPU, 64 RAM, 40 Tb HDD)
> 1 ES master (32 CPU, 64 RAM, 1 Tb HDD)
> 1 Kibana (16 CPU, 64 RAM, 1 Tb HDD)
>
>
>
