Hi there, I configured an IBM X3650 M4 for development and testing purposes. It’s composed by: - 2 x Intel Xeon E5-2690 @ 2.90Ghz (2 x 8 physical Cores + HT) - 96GB RAM DDR3 1333MHz (12 x 8GB) - 2 x 146GB SAS HDDs @ 15k rpm configured in RAID1 (mdadm) - 6 x 525GB SATA SSDs (over-provisioned at 25%, so 393GB available)
I’ve done a lot of testing focusing on 4k and 8k workloads and found that the IOPS of those SSDs are half the expected. On serverfault.com someone suggested me that probably the bottle neck is the embedded RAID controller, a IBM ServeRaid m5110e, which mounts a LSI 2008 controller. I’m using the disks in JBOD mode with mdadm software RAID, which is blazing fast. The CPU is also very fast, so I don’t mind having a little overhead due to software RAID. My typical workload is Postgres run as a DWH with 1 to 2 billions of rows, big indexes, partitions and so on, but also intensive statistical computations. Here’s my post on serverfault.com ( http://serverfault.com/questions/833642/slow-ssd-performance-ibm-x3650-m4-7915 ) and here’s a graph of those six SSDs evaluated using fio as stand-alone disks (outside of the RAID): All those IOPS should be doubled if all was working correctly. The curve trend is correct for increasing IO Depths. Anyway, I would like to buy a HBA controller that leverages those 6 SSDs. Each SSD should deliver about 80k to 90k IOPS, so in RAID10 I should get ~240k IOPS (6 x 80k / 2) and in RAID0 ~480k IOPS (6 x 80k). I’ve seen that mdadm effectively scales performance, but the controller limits the overal IOPS at ~120k (exactly the half of the expected IOPS). What HBA controller would you suggest me able to handle 500k IOPS? My server is able to handle 8 more SSDs, for a total of 14 SSDs and 1260k theoretical IOPS. If we imagine adding only 2 more disks, I will achieve 720k theoretical IOPS in RAID0. What HBA controller would you suggest me able to handle more than 700k IOPS? Have you got some advices about using mdadm RAID software on SATAIII SSDs and plain HBA? Thank you everyone Pietro Pugni
