> The only reason setting mode=off is the observed ping-pong.
Ah, I didn't catch that you were actively experiencing that. > The index pool is small (3 GiB stored) and PG merge/split is fast (~3 > minutes). Right after adjusting pg_num (to 32/64) the health warning flips > into the opposite, several times, back and forth. Do you set the mode to on for all pools? Does every pool use a CRUSH rule that specifies a device class? > The SSD-OSD's size is only 32 GiB, 4 per host and 3 hosts. The small size and > count was chosen out of unawareness (pre-created LVs). Remind me please, how many OSDs total? The index pool does benefit from more vs larger OSDs, though if you're shaving that device class with other purposes it can be nuanced. If > Could this be something that confuses the autoscaler? For sure the autoscaler isn't great at very small clusters in some cases, but I wouldn't expected flapping. > >> What you're thinking of is the per-pool threshold. Historically this has >> defaulted to 3.0 : the autoscaler won't make a change until the calculated >> value varies by more than a factor of 3 from the current setting. I think >> of this as hysteresis, but there's probably a better term. There's been >> some fairly recent work to do this more surgically, which I _think_ removes >> the per-pool threshold, but in any event, I believe that recent autoscaler >> releases are fairly immune to this flapping. > > Thanks for clarification on BIAS and threshold. Yes I meant threshold, but > looks like threshold is obsolete long ago (can't find it in the docs[1])? Curious, could be that we never documented it upstream. IBM FWIW documents this: https://www.ibm.com/docs/en/storage-ceph/9.9.0?topic=groups-configuring-placement-group-auto-scaling (IBM 9 is Tentacle, 8 is Squid) > At least for me the autoscaler seems not immune to flapping. What's your `ceph df`? Do you set any of the knobs to non-default values? I would think that your buckets.data pool would show a higher RATIO, but the bulk flag on it may be a factor. I've yet to fully wrap my head around how the bulk flag actually works, but it might be interesting to set it on the index pool too to see what happens with the autoscaler enabled. You might try # ceph osd pool set default.rgw.buckets.index pg_num_min 64 > > Thanks, Sascha. > > [1] > https://docs.ceph.com/en/latest/rados/operations/pools/#setting-pool-values > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
