Re: [ceph-users] Number of PGs: fix from start or change as we grow ?

[Christian Balzer]

Hello,

On Wed, 3 Aug 2016 15:15:21 +0300 Maged Mokhtar wrote:

> Hello,
> 
> I would like to build a small cluster with 20 disks to start but in the 
> future would like to gradually increase it to maybe 200 disks.
> Is it better to fix the number of PGs in the pool from the beginning or is it 
> better to start with a small number and then gradually change the number of 
> PGs as the system grows ?
> Is the act of changing the number of PGs in a running cluster something that 
> can be done regularly ? 
> 

This is both something that's strongly hinted at in the documentation as
well as discussed countless times on this ML (google is your friend),
along with the means to minimize the impact of this action.

Setting the "correct" PG value for a 200 OSD cluster (8192) at the start
with 20 OSDs (recommended value 512) is not going to be pretty and will
have your cluster in a warning state with about 1200 (!) PGs per OSD at
the very least.
Never mind CPU and RAM usage. 

Increasing PGs is an involved and costly operation, so it should be done
as little as possible. 
However if your cluster is designed/configured well and not operating
constantly at its breaking point, it's also an operation that should be
doable w/o major impacts. 

I'd start with 1024 PGs on those 20 OSDs, at 50 OSDs go to 4096 PGs and at
around 100 OSDs it is safe to go to 8192 PGs.

Christian
-- 
Christian BalzerNetwork/Systems Engineer
ch...@gol.com   Global OnLine Japan/Rakuten Communications
http://www.gol.com/
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Number of PGs: fix from start or change as we grow ?

[Luis Periquito]

Changing the number of PGs is one of the most expensive operations you
can run, and should be avoided as much as possible.

Having said that you should try to avoid having way too many PGs with
very few OSDs, but it's certainly preferable to splitting PGs...

On Wed, Aug 3, 2016 at 1:15 PM, Maged Mokhtar
 wrote:
> Hello,
>
> I would like to build a small cluster with 20 disks to start but in the
> future would like to gradually increase it to maybe 200 disks.
> Is it better to fix the number of PGs in the pool from the beginning or is
> it better to start with a small number and then gradually change the number
> of PGs as the system grows ?
> Is the act of changing the number of PGs in a running cluster something that
> can be done regularly ?
>
> Cheers
> /Maged
>
>
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Number of PGs: fix from start or change as we grow ?

[Maged Mokhtar]

Hello,

I would like to build a small cluster with 20 disks to start but in the future 
would like to gradually increase it to maybe 200 disks.
Is it better to fix the number of PGs in the pool from the beginning or is it 
better to start with a small number and then gradually change the number of PGs 
as the system grows ?
Is the act of changing the number of PGs in a running cluster something that 
can be done regularly ? 

Cheers 
/Maged



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com