[ceph-users] Re: Building ceph clusters with 8TB SSD drives?

2021-05-10 Thread Erik Lindahl
Hi Matt, Yes, we've experimented a bit with consumer SSDs, and also done some benchmarks. The main reason for SSDs is typically to improve IOPS for small writes, since even HDDs will usually give you quite good aggregated bandwidth as long as you have enough of them - but for high-IOPS usage most

[ceph-users] Re: Building ceph clusters with 8TB SSD drives?

2021-05-08 Thread Erik Lindahl
The main reason for SSDs is typically to improve IOPS for small writes, but for that usage most (all) consumer SSDs we have tested perform badly in Ceph. The reason for this is that Ceph requires SYNC writes, and since consumer SSDs (and now even some cheap datacenter ones) don't have capacitors

[ceph-users] Re: Building ceph clusters with 8TB SSD drives?

2021-05-07 Thread Frank Schilder
We are using 4TB Kingston DC500M drives for 6+2 EC RBD data pools, 2 OSDs per disk. They deliver great iops, but I think they are TLC, so probably not comparable with QLC drives. I think QLC drives are OK for mostly cold/static data due to their performance drop when running full: https://www.h

[ceph-users] Re: Building ceph clusters with 8TB SSD drives?

2021-05-07 Thread Nico Schottelius
Matt Larson writes: > Is anyone trying Ceph clusters containing larger (4-8TB) SSD drives? > > 8TB SSDs are described here ( > https://www.anandtech.com/show/16136/qlc-8tb-ssd-review-samsung-870-qvo-sabrent-rocket-q > ) and make use QLC NAND flash memory to reach the costs and capacity. > Curre