Isn’t it too much for ssd 4 osd? Normally nvme is suitable for 4osd isn’t it?
Istvan Szabo Senior Infrastructure Engineer --------------------------------------------------- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com<mailto:istvan.sz...@agoda.com> --------------------------------------------------- On 2021. Oct 26., at 10:23, Frank Schilder <fr...@dtu.dk> wrote: Email received from the internet. If in doubt, don't click any link nor open any attachment ! ________________________________ It looks like the bottleneck is the bstore_kv_sync thread, there seems to be only one running per OSD daemon independent of shard number. This would imply a rather low effective queue depth per OSD daemon. Are there ways to improve this other than deploying even more OSD daemons per OSD? Thanks! ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Frank Schilder <fr...@dtu.dk> Sent: 26 October 2021 09:41:44 To: ceph-users Subject: [ceph-users] ceph-osd iodepth for high-performance SSD OSDs Hi all, we deployed a pool with high-performance SSDs and I'm testing aggregated performance. We seem to hit a bottleneck that is not caused by drive performance. My best guess at the moment is, that the effective iodepth of the OSD daemons is too low for these drives. I have 4 OSDs per drive and I vaguely remember that there are parameters to modify the degree of concurrency an OSD daemon uses to write to disk. Are these parameters the ones I'm looking for: "osd_op_num_shards": "0", "osd_op_num_shards_hdd": "5", "osd_op_num_shards_ssd": "8", "osd_op_num_threads_per_shard": "0", "osd_op_num_threads_per_shard_hdd": "1", "osd_op_num_threads_per_shard_ssd": "2", How do these apply if I have these drives in a custom device class rbd_perf? Could I set, for example ceph config set osd/class:rbd_perf osd_op_num_threads_per_shard 4 to increase concurrency on this particular device class only? Is it possible to increase the number of shards at run-time? Thanks for your help! Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
