In fact, when you increase your pg number, the new pgs will have to peer
first and during this time, a lot a pg will be unreachable. The best way to
upgrade the number of PG of a cluster (you 'll need to adjust the number of
PGP too) is :

   - Don't forget to apply Goncalo advices to keep your cluster responsive
   for client operations. Otherwise, all the IO and CPU will be used for the
   recovery operations and your cluster will be unreachable. Be sure that all
   these new parameters are in place before upgrading your cluster

   - stop and wait for scrub and deep-scrub operations

ceph osd set noscrub
ceph osd set nodeep-scrub

   - set you cluster in maintenance mode with :

ceph osd set norecover
ceph osd set nobackfill
ceph osd set nodown
ceph osd set noout

wait for your cluster not have scrub or deep-scrub opration anymore

   - upgrade the pg number with a small increment like 256

   - wait for the cluster to create and peer the new pgs (about 30 seconds)

   - upgrade the pgp number with the same increment

   - wait for the cluster to create and peer (about 30 seconds)

(Repeat the last 4 operations until you reach the number of pg and pgp you

At this time, your cluster is still functionnal.

   - Now you have to unset the maintenance mode

ceph osd unset noout
ceph osd unset nodown
ceph osd unset nobackfill
ceph osd unset norecover

It will take some time to replace all the pgs but at the end you will have
a cluster with all pgs active+clean.During all the operation,your cluster
will still be functionnal if you have respected Goncalo parameters

   - When all the pgs are active+clean, you can re-enable the scrub and
   deep-scrub operations

ceph osd unset noscrub
ceph osd unset nodeep-scrub
ceph-users mailing list

Reply via email to