Considering
https://github.com/ceph/ceph/blob/f6edcef6efe209e8947887752bd2b833d0ca13b7/src/osd/OSD.cc#L10086,
the OSD:
- always sets and updates its per osd osd_mclock_max_capacity_iops_{hdd,ssd}
when the benchmark occurs and its measured iops is below or equal to
osd_mclock_iops_capacity_threshold_{hdd,ssd}
but
- doesn't remove osd_mclock_max_capacity_iops_{hdd,ssd} when the measured iops
exceeds osd_mclock_iops_capacity_threshold_{hdd,ssd} (500 for HDD and 80.000
for SSD) and the current value for osd_mclock_max_capacity_iops_{hdd,ssd} is
set below its default (315 for HDD and 21500 for SSD)
Thus per osd osd_mclock_max_capacity_iops_hdd sometimes being set as low as
0.145327 (as per Michel's post) and never being updated afterwards leading to
performance issues.
The idea of a minimum threshold below which
osd_mclock_iops_capacity_threshold_{hdd,ssd} should not be set seems relevant.
CC'ing Sridhar to have his thoughts.
Cheers,
Frédéric.
----- Le 22 Mar 24, à 19:37, Kai Stian Olstad ceph+l...@olstad.com a écrit :
> On Fri, Mar 22, 2024 at 06:51:44PM +0100, Frédéric Nass wrote:
>>
>>> The OSD run bench and update osd_mclock_max_capacity_iops_{hdd,ssd} every
>>> time
>>> the OSD is started.
>>> If you check the OSD log you'll see it does the bench.
>>
>>Are you sure about the update on every start? Does the update happen only if
>>the
>>benchmark result is < 500 iops?
>>
>>Looks like the OSD does not remove any set configuration when the benchmark
>>result is > 500 iops. Otherwise, the extremely low value that Michel reported
>>earlier (less than 1 iops) would have been updated over time.
>>I guess.
>
> I'm not completely sure, it's a couple a month since I used mclock, have
> switch
> back to wpq because of a nasty bug in mclock that can freeze cluster I/O.
>
> It could be because I was testing osd_mclock_force_run_benchmark_on_init.
> The OSD had DB on SSD and data on HDD, so the measured to about 1700 IOPS and
> was ignored because of the 500 limit.
> So only the SSD got the osd_mclock_max_capacity_iops_ssd set.
>
> --
> Kai Stian Olstad
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
