Hi, It is indeed possible that mapping fails if there are just enough racks to match the constraint. And the probability of a bad mapping increases when the number of PG increases because there is a need for more mapping. You can tell crush to try harder with
step set_chooseleaf_tries 10 Be careful though : increasing this number will change mapping. It will not just fix the bad mappings you're seeing, it will also change the mappings that succeeded with a lower value. Once you've set this parameter, it cannot be modified. Would you mind sharing the erasure code profile you plan to work with ? Cheers On 09/09/2014 12:39, Lei Dong wrote: > Hi ceph users: > > I want to create a customized crush rule for my EC pool (with replica_size = > 11) to distribute replicas into 6 different Racks. > > I use the following rule at first: > > Step take default // root > Step choose firstn 6 type rack// 6 racks, I have and only have 6 racks > Step chooseleaf indep 2 type osd // 2 osds per rack > Step emit > > I looks fine and works fine when PG num is small. > But when pg num increase, there are always some PGs which can not take all > the 6 racks. > It looks like “Step choose firstn 6 type rack” sometimes returns only 5 racks. > After some investigation, I think it may caused by collision of choices. > > Then I come up with another solution to solve collision like this: > > Step take rack0 > Step chooseleaf indep 2 type osd > Step emit > Step take rack1 > …. > (manually take every rack) > > This won’t cause rack collision, because I specify rack by name at first. But > the problem is that osd in rack0 will always be primary osd because I choose > from rack0 first. > > So the question is what is the recommended way to meet such a need > (distribute 11 replicas into 6 racks evenly in case of rack failure)? > > > Thanks! > LeiDong > > > > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Loïc Dachary, Artisan Logiciel Libre
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
