Good evening,
I've to tackle an old, probably recurring topic: HBAs vs. Raid
controllers. While generally speaking many people in the ceph field
recommend to go with HBAs, it seems in our infrastructure the only
server we phased in with an HBA vs. raid controller is actually doing
worse in terms of latency.
For the background: we have many Perc H800+MD1200 [1] systems running with
10TB HDDs (raid0, read ahead, writeback cache).
One server has LSI SAS3008 [0] instead of the Perc H800,
which comes with 512MB RAM + BBU. On most servers latencies are around
4-12ms (average 6ms), on the system with the LSI controller we see
20-60ms (average 30ms) latency.
Now, my question is, are we doing some inherently wrong with the
SAS3008 or does in fact the cache help to possible reduce seek time?
We were considering to move more towards LSI HBAs to reduce maintenance
effort, however if we have a factor of 5 in latency between the two
different systems, it might be better to stay on the H800 path for
disks.
Any input/experiences appreciated.
Best regards,
Nico
[0]
05:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS3008
PCI-Express Fusion-MPT SAS-3 (rev 02)
Subsystem: Dell 12Gbps HBA
Kernel driver in use: mpt3sas
Kernel modules: mpt3sas
[1]
08:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2108
[Liberator] (rev 05)
Subsystem: Dell PERC H800 Adapter
Kernel driver in use: megaraid_sas
Kernel modules: megaraid_sas
--
Sustainable and modern Infrastructures by ungleich.ch
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
