The extra pools are probably the data and metadata pools that are automatically created for cephfs.
http://ceph.com/pgcalc/ is a useful tool for helping to work out how many PGs your pools should have. Rich On 04/05/17 15:41, David Turner wrote: > I'm guessing you have more than just the 1 pool with 128 PGs in your > cluster (seeing as you have 320 PGs total, I would guess 2 pools with > 128 PGs and 1 pool with 64 PGs). The combined total number of PGs for > all of your pools is 320 and with only 3 OSDs and most likely replica > size 3... that leaves you with too many (320) PGs per OSD. This will > not likely affect your testing, but if you want to fix the problem you > will need to delete and recreate your pools with a combined lower > total number of PGs. > > The number of PGs is supposed to reflect how much data each pool is > going to have. If you have 1 pool that will have 75% of your > cluster's data, another pool with 20%, and a third pool with 5%... > then the number of PGs they have should reflect that. Based on trying > to have somewhere between 100-200 PGs per osd, and the above > estimation for data distribution, you should have 128 PGs in the first > pool, 32 PGs in the second, and 8 PGs in the third. Each OSD would > have 168 PGs and each PG will be roughly the same size between each > pool. If you were to add more OSDs, then you would need to increase > those numbers to account for the additional OSDs to maintain the same > distribution. The above math is only for 3 OSDs. If you had 6 OSDs, > then the goal would be to have somewhere between 200-400 PGs total to > maintain the same 100-200 PGs per OSD. > > On Thu, May 4, 2017 at 10:24 AM psuresh <[email protected] > <mailto:[email protected]>> wrote: > > Hi, > > I'm running 3 osd in my test setup. I have created PG pool with > 128 as per the ceph documentation. > But i'm getting too many PGs warning. Can anyone clarify? why > i'm getting this warning. > > Each OSD contain 240GB disk. > > cluster 9d325da2-3d87-4b6b-8cca-e52a4b65aa08 > health HEALTH_WARN > *too many PGs per OSD (320 > max 300)* > monmap e2: 3 mons at > {dev-ceph-mon1:6789/0,dev-ceph-mon2:6789/0,dev-ceph-mon3:6789/0} > election epoch 6, quorum 0,1,2 > dev-ceph-mon1,dev-ceph-mon2,dev-ceph-mon3 > fsmap e40: 1/1/1 up {0=dev-ceph-mds-active=up:active} > osdmap e356: 3 osds: 3 up, 3 in > flags sortbitwise,require_jewel_osds > pgmap v32407: 320 pgs, 3 pools, 27456 MB data, 220 kobjects > 100843 MB used, 735 GB / 833 GB avail > 320 active+clean > > Regards, > Suresh >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
