Hi Roderick, While we don’t process that much, only 4 Gbps and 500k pps peak, I
have it running on an old core 2 duo with 6G of RAM. Yes, it is an old desktop
system (8 years old). Nfsen uses barely any resources. I think your server is
fine with the exception of storage. How long do you want to keep the flows?
Your cache is only used for those queries where you are frequently accessing
the data. Therefore, you will not see any speed increase with the SSD caching
when searching for random data. I would highly recommend either 10/15k SAS or
better yet, SSDs. Also look at RAID 10 for speed. If you are doing RAID 6,
might as well do RAID 10. If cost is an issue, I would decrease the RAM and
increase your storage. Also with data that important, make sure you get a good
RAID card
http://www.avagotech.com/products/storage/raid-controllers/megaraid-sas-9361-16i.
If you just use a simple RAID card without a battery and large cache, you may
corrupt your data in the event of a power failure.
Greg Williams, ME
Director of Networks and Infrastructure
Information Technology
University of Colorado Colorado Springs
1420 Austin Bluffs Parkway, (EPC 136A)
Colorado Springs, CO 80918
www.uccs.edu<http://www.uccs.edu/>
From: Roderick Mooi [mailto:roder...@sanren.ac.za]
Sent: Tuesday, November 1, 2016 7:17 AM
To: nfdump-disc...@lists.sourceforge.net; nfsen-discuss@lists.sourceforge.net
Cc: joh...@sanren.ac.za; Ishmael Tsoaela <ishm...@sanren.ac.za>
Subject: [Nfsen-discuss] Netflow collector hardware advice
Good day
We would like to procure a server for netflow data collection and processing
(nfdump + nfsen) and are looking for some hardware advice if you could please
assist?
Requirement:
Import flow data from 4 x 10 Gbps router interfaces
- currently we have 3 active interfaces with 5-6 Gbps of peak traffic each and
between 500k and 1mil pps each
- Juniper MX480 and Cisco ASR 9010
- unsampled (full)
Execute queries in a reasonable amount of time (ideally no more than a few
minutes but I know it depends on the complexity and time frame which will vary
depending on our use case at the time)
Spec so far:
Supermicro SSG-6028R-E1CR24N chassis
Intel CPU: 4C E5-2623V4 2.6G 10M 8GT QPI
128GB DDR4 RAM [expandable]
60TB storage (10x6TB SAS 12Gb/s) RAID6 [expandable]
2 x 200GB SSD caching drives with LSI Cachecade Pro 2.0
Questions:
CPU
- which is better for nfdump+nfsen - more cores or more speed? Or pros and cons
of each? i.e. would a 10C E5-2630V4 2.2G 25M 8GT QP be a better or worse choice?
Memory
- how much is needed? (I figure it depends on the queries, so let's say we're
looking for any connections containing a certain destination IP address within
a 1 month period)
Storage
- we'd like to keep at least 3 months data for 4x10Gbps interfaces unsampled.
How much do we need?
- 4TB or 6TB drives (Seagate Makara)
- read cache only?
Thanks very much,
Roderick
--
Engineer
South African National Research Network (SANReN)
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
