Re: [ceph-users] CRUSHMAP for chassis balance
Hi Gregory,
Thanks for the direction that I finish with 3 different rule in a ruleset for
different rep size:
Tested no bad-mapping and host / osd are correctly balanced between 2 chassis.
Not sure if it can be optimized but I am happy with current result:
rule rule_rep2 {
ruleset 0
type replicated
min_size 2
max_size 2
step take chassis1
step chooseleaf firstn 1 type host
step emit
step take chassis2
step chooseleaf firstn 1 type host
step emit
}
rule rule_rep34 {
ruleset 0
type replicated
min_size 3
max_size 4
step take default
step choose firstn 2 type chassis
step chooseleaf firstn 2 type host
step emit
}
rule rule_rep56 {
ruleset 0
type replicated
min_size 5
max_size 6
step take default
step choose firstn 3 type chassis
step chooseleaf firstn 3 type host
step emit
}
Luke
From: Gregory Farnum [mailto:g...@gregs42.com]
Sent: Friday, February 13, 2015 11:01 PM
To: Luke Kao; ceph-users@lists.ceph.com
Subject: Re: [ceph-users] CRUSHMAP for chassis balance
With sufficiently new CRUSH versions (all the latest point releases on LTS?) I
think you can simply have the rule return extra IDs which are dropped if they
exceed the number required. So you can choose two chassis, then have those both
choose to lead OSDs, and return those 4 from the rule.
-Greg
On Fri, Feb 13, 2015 at 6:13 AM Luke Kao
luke@mycom-osi.commailto:luke@mycom-osi.com wrote:
Dear cepher,
Currently I am working on crushmap to try to make sure the at least one copy
are going to different chassis.
Say chassis1 has host1,host2,host3, and chassis2 has host4,host5,host6.
With replication =2, it’s not a problem, I can use the following step in rule
step take chasses1
step chooseleaf firstn 1 type host
step emit
step take chasses2
step chooseleaf firstn 1 type host
step emit
But for replication=3, I tried
step take chasses1
step chooseleaf firstn 1 type host
step emit
step take chasses2
step chooseleaf firstn 1 type host
step emit
step take default
step chooseleaf firstn 1 type host
step emit
At the end, the 3rd osd returned in rule test is always duplicate with first 1
or first 2.
Any idea or what’s the direction to move forward?
Thanks in advance
BR,
Luke
MYCOM-OSI
This electronic message contains information from Mycom which may be privileged
or confidential. The information is intended to be for the use of the
individual(s) or entity named above. If you are not the intended recipient, be
aware that any disclosure, copying, distribution or any other use of the
contents of this information is prohibited. If you have received this
electronic message in error, please notify us by post or telephone (to the
numbers or correspondence address above) or by email (at the email address
above) immediately.
___
ceph-users mailing list
ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
This electronic message contains information from Mycom which may be privileged
or confidential. The information is intended to be for the use of the
individual(s) or entity named above. If you are not the intended recipient, be
aware that any disclosure, copying, distribution or any other use of the
contents of this information is prohibited. If you have received this
electronic message in error, please notify us by post or telephone (to the
numbers or correspondence address above) or by email (at the email address
above) immediately.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] CRUSHMAP for chassis balance
Dear cepher, Currently I am working on crushmap to try to make sure the at least one copy are going to different chassis. Say chassis1 has host1,host2,host3, and chassis2 has host4,host5,host6. With replication =2, it's not a problem, I can use the following step in rule step take chasses1 step chooseleaf firstn 1 type host step emit step take chasses2 step chooseleaf firstn 1 type host step emit But for replication=3, I tried step take chasses1 step chooseleaf firstn 1 type host step emit step take chasses2 step chooseleaf firstn 1 type host step emit step take default step chooseleaf firstn 1 type host step emit At the end, the 3rd osd returned in rule test is always duplicate with first 1 or first 2. Any idea or what's the direction to move forward? Thanks in advance BR, Luke MYCOM-OSI This electronic message contains information from Mycom which may be privileged or confidential. The information is intended to be for the use of the individual(s) or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or any other use of the contents of this information is prohibited. If you have received this electronic message in error, please notify us by post or telephone (to the numbers or correspondence address above) or by email (at the email address above) immediately. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] CRUSHMAP for chassis balance
With sufficiently new CRUSH versions (all the latest point releases on LTS?) I think you can simply have the rule return extra IDs which are dropped if they exceed the number required. So you can choose two chassis, then have those both choose to lead OSDs, and return those 4 from the rule. -Greg On Fri, Feb 13, 2015 at 6:13 AM Luke Kao luke@mycom-osi.com wrote: Dear cepher, Currently I am working on crushmap to try to make sure the at least one copy are going to different chassis. Say chassis1 has host1,host2,host3, and chassis2 has host4,host5,host6. With replication =2, it’s not a problem, I can use the following step in rule step take chasses1 step chooseleaf firstn 1 type host step emit step take chasses2 step chooseleaf firstn 1 type host step emit But for replication=3, I tried step take chasses1 step chooseleaf firstn 1 type host step emit step take chasses2 step chooseleaf firstn 1 type host step emit step take default step chooseleaf firstn 1 type host step emit At the end, the 3rd osd returned in rule test is always duplicate with first 1 or first 2. Any idea or what’s the direction to move forward? Thanks in advance BR, Luke MYCOM-OSI -- This electronic message contains information from Mycom which may be privileged or confidential. The information is intended to be for the use of the individual(s) or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or any other use of the contents of this information is prohibited. If you have received this electronic message in error, please notify us by post or telephone (to the numbers or correspondence address above) or by email (at the email address above) immediately. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
