The placement of PGs is random in the cluster and takes into account any
CRUSH rules which may also skew the distribution. Having more PGs will help
give more options for placing PGs, but it still may not be adequate. It is
recommended to have between 100-150 PGs per OSD, and you are pretty close.
If you aren't planning to add any more pools, then splitting the PGs for
pools that have a lot of data can help.
To get things to be more balanced, you can reweight the high utlization
OSDs down to cause CRUSH to migrate some PGs off. This won't mean that they
will get moved to the lowest utilized OSDs (they might wind up on another
one that is pretty full). So, it may take several iterations to get things
balanced. Just be sure that if you reweighted one down and it is now much
lower usage than the others to reweight it back up to attract some PGs back
to it.
```ceph osd reweight {osd-num} {weight}```
----------------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
On Mon, Jun 24, 2019 at 2:25 AM [email protected] <
[email protected]> wrote:
> Hello everyone,
>
> We have some osd on the ceph.
> Some osd's usage is more than 77% and another osd's usage is 39% in the
> same host.
>
> I wonder why osd’s usage is different.(Difference is large) and how can i
> fix it?
>
> ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS TYPE
> NAME
> -2 93.26010 - 93.3TiB 52.3TiB 41.0TiB 56.04 0.98 -
> host serverA
> …...
> 33 HDD 9.09511 1.00000 9.10TiB 3.55TiB 5.54TiB 39.08 0.68 66
> osd.4
> 45 HDD 7.27675 1.00000 7.28TiB 5.64TiB 1.64TiB 77.53 1.36 81
> osd.7
> …...
>
> -5 79.99017 - 80.0TiB 47.7TiB 32.3TiB 59.62 1.04 -
> host serverB
> 1 HDD 9.09511 1.00000 9.10TiB 4.79TiB 4.31TiB 52.63 0.92 87
> osd.1
> 6 HDD 9.09511 1.00000 9.10TiB 6.62TiB 2.48TiB 72.75 1.27 99
> osd.6
> …...
>
> Thank you
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
