Hi all,
We got 4 PG active+remapped in our cluster. We set the pool’s ruleset to
ruleset 0 and got HEALTH_OK. After we set the ruleset to ruleset 1, 4 pg is
active+remapped. The testing result from crushtool also shows some bad mapping
exists. Anyone happened to know the reason?
pool 0 'rbd' replicated size 3 min_size 2 crush_ruleset 1 object_hash rjenkins
pg_num 256 pgp_num 256 last_change 421 flags hashpspool stripe_width 0
[root@Ceph-Stress-01 ~]# ceph pg ls remapped
PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG
DISK_LOG STATE STATE_STAMP VERSION REPORTED UP
UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP
LAST_DEEP_SCRUB DEEP_SCRUB_STAMP
0.33 0 0 0 0 0 0 0
0 active+remapped 2017-03-07 22:53:02.453447 0'0 419:110 [3,22] 3
[3,22,4] 3 0'0 2017-03-07 13:29:32.110280 0'0
2017-03-07 13:29:32.110280
0.3b 0 0 0 0 0 0 0
0 active+remapped 2017-03-07 22:53:02.619526 0'0 419:110 [2,20] 2
[2,20,17] 2 0'0 2017-03-07 13:29:32.110287 0'0
2017-03-07 13:29:32.110287
0.49 0 0 0 0 0 0 0
0 active+remapped 2017-03-07 22:53:02.453239 0'0 419:104 [4,20] 4
[4,20,19] 4 0'0 2017-03-07 13:29:32.110257 0'0
2017-03-07 13:29:32.110257
0.54 0 0 0 0 0 0 0
0 active+remapped 2017-03-07 22:53:02.101725 0'0 419:207 [19,3] 19
[19,3,20] 19 0'0 2017-03-07 13:29:32.110262 0'0
2017-03-07 13:29:32.110262
rule replicated_ruleset_one_host {
ruleset 0
type replicated
min_size 1
max_size 10
step take default
step choose firstn 0 type osd
step emit
}
rule replicated_ruleset_multi_host {
ruleset 1
type replicated
min_size 1
max_size 10
step take default
step chooseleaf firstn 0 type host
step emit
}
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 15.00000 root default
-2 7.00000 host Ceph-Stress-01
0 1.00000 osd.0 up 1.00000 1.00000
1 1.00000 osd.1 up 1.00000 1.00000
2 1.00000 osd.2 up 1.00000 1.00000
3 1.00000 osd.3 up 1.00000 1.00000
4 1.00000 osd.4 up 1.00000 1.00000
5 1.00000 osd.5 up 1.00000 1.00000
6 1.00000 osd.6 up 1.00000 1.00000
-4 7.00000 host Ceph-Stress-03
16 1.00000 osd.16 up 1.00000 1.00000
17 1.00000 osd.17 up 1.00000 1.00000
18 1.00000 osd.18 up 1.00000 1.00000
19 1.00000 osd.19 up 1.00000 1.00000
20 1.00000 osd.20 up 1.00000 1.00000
21 1.00000 osd.21 up 1.00000 1.00000
22 1.00000 osd.22 up 1.00000 1.00000
-3 1.00000 host Ceph-Stress-02
7 1.00000 osd.7 up 1.00000 1.00000
[root@Ceph-Stress-02 ~]# crushtool --test -i crushmap --rule 1 --min-x 1
--max-x 5 --num-rep 3 --show-utilization --show-mappings --show-bad-mappings
rule 1 (replicated_ruleset_multi_host), x = 1..5, numrep = 3..3
CRUSH rule 1 x 1 [5,7,21]
CRUSH rule 1 x 2 [17,6]
bad mapping rule 1 x 2 num_rep 3 result [17,6]
CRUSH rule 1 x 3 [19,7,1]
CRUSH rule 1 x 4 [5,22,7]
CRUSH rule 1 x 5 [21,0]
bad mapping rule 1 x 5 num_rep 3 result [21,0]
rule 1 (replicated_ruleset_multi_host) num_rep 3 result size == 2: 2/5
rule 1 (replicated_ruleset_multi_host) num_rep 3 result size == 3: 3/5
device 0: stored : 1 expected : 1
device 1: stored : 1 expected : 1
device 5: stored : 2 expected : 1
device 6: stored : 1 expected : 1
device 7: stored : 3 expected : 1
device 17: stored : 1 expected : 1
device 19: stored : 1 expected : 1
device 21: stored : 2 expected : 1
device 22: stored : 1 expected : 1
Thanks,
Tim
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
