I have been doing a lot of hardware+Metron work lately as I prepare to buy my prod hardware, I would be happy to work with you on things. Once my build is in production I will publish statistics regarding my environment. For some very brief mobile friendly metrics, I ingest about 25,000 events per second/600GB of uncompressed logs a day and I'm looking at a 16 DN cluster with 256 GB RAM (8x32GB for later expansion to 512GB), 9 non-OS spinning drives, and 2x2660v4 CPUs. Either 4 or 6 similar boxes (but with 6 spinning disks non-OS) for search. This doesn't include things like NN, gateway boxes, etc.
Happy to talk more detail off list, until I can get more formal approval to release details publicly. Do you have a general idea of logs per second/uncompressed log size per day? Will you be storing PCAP? Any complicated enrichments/triage/transformations? Modeling is another huge curveball. Jon On Mon, Mar 20, 2017, 5:33 PM James Sirota <jsir...@apache.org> wrote: > Hi Laurens, this depends on how many sources you ingest (and what these > sources are), how many enrichments you apply, the number of triage rules > you use, number of models you deploy, etc. There is not really a number > that we can give you. Can you describe your use case for us? > > 20.03.2017, 08:57, "Laurens Vets" <laur...@daemon.be>: > > Hi List, > > > > Does anyone have any hardware requirements and/or performance numbers > > they can share? I know that this is a very open question, but anything > > would help :) We're basically starting from 0. Thanks! > > > > -Laurens > > ------------------- > Thank you, > > James Sirota > PPMC- Apache Metron (Incubating) > jsirota AT apache DOT org > -- Jon
