On Tue, Feb 2, 2016 at 5:19 AM, Tinker <[email protected]> wrote: > 1) I need some SSD storage but don't like that it could break together - I > mean, a bug in your system will feed your SSD at full bandwidth for ~7h-7 > days, it's completely fried - that's not OK, so putting a "redundance layer" > in the from of an underlying magnet storage layer is really justified. > > 2) I need some bulk storage, and I want the terabytes to be really cheap so > that i NEVER will run out of archival space. An 8TB magnet HDD costs in the > range USD 500.
Last month I purchased another 400 TB of magnetic and SSD storage, so with that recent research in mind: * SSD models are differentiated by write endurance, interface (SATA/SAS vs NVMe), and consumer vs data centre grade. The latter offer continuous (24x7) workload latency guarantees and work well in RAID configurations as there is minimal latency variability between units. I'm unsure of the OpenBSD NVMe status, but you need NVMe if you want to pull the rated ~2500 MB/sec out of NVMe units. Sticking with SATA/SAS SSD means closer to 1/5th that maximum read performance, although that may well be enough (and will save you a lot of money, too). I'd avoid doing RAID with consumer grade SSDs if you have high throughput expectations. * Some magnetic drives are now also being rated with annual workload limits and have associated warranty implications. See for example http://www.seagate.com/au/en/products/enterprise-servers-storage/nearline-storage/. Be careful to review the data sheets so you don't have future warranty claims rejected. I've found it's easy to KISS by grouping by file type (eg immutable or mutable or append-only, sequentially accessed vs randomly accessed, main reader bottleneck being CPU/RAM or IO etc) and putting them in optimal primary and backup locations accordingly. I find the idea of transparent storage tiers undesirable, as there's no way I can reason about that and guarantee consistent throughput and latency results (let alone how to recover when a few drives out of the 60+ fail, or a non-JBOD controller dies). Are you able to share more about your storage requirements (capacity, sequential throughput, IO latency, resiliency, target chassis details etc) so we can offer suggestions?
