I suppose I may have found the solution I was unaware existed.
> balancer optimize <plan> {<pools> [<pools>...]} : Run optimizer to create a
> new plan
So apparently you can create a plan specific to a pool(s).
So just to double check this, I created two plans, plan1 with the hdd pool (and
not the ssd pool); plan2 with no arguments.
I then ran ceph balancer show planN and also ceph osd crush weight-set dump.
Then compared the values in the weight-set dump against the values in the two
plans, and concluded that plan1 did not adjust the values for ssd osd’s, which
is exactly what I was looking for:
> ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
> -14 17.61093 host ceph00
> 24 ssd 1.76109 osd.24 up 1.00000 1.00000
> 25 ssd 1.76109 osd.25 up 1.00000 1.00000
> 26 ssd 1.76109 osd.26 up 1.00000 1.00000
> 27 ssd 1.76109 osd.27 up 1.00000 1.00000
> 28 ssd 1.76109 osd.28 up 1.00000 1.00000
> 29 ssd 1.76109 osd.29 up 1.00000 1.00000
> 30 ssd 1.76109 osd.30 up 1.00000 1.00000
> 31 ssd 1.76109 osd.31 up 1.00000 1.00000
> 32 ssd 1.76109 osd.32 up 1.00000 1.00000
> 33 ssd 1.76109 osd.33 up 1.00000 1.00000
ceph osd crush weight-set dump
> {
> "bucket_id": -14,
> "weight_set": [
> [
> 1.756317,
> 1.613647,
> 1.733200,
> 1.735825,
> 1.961304,
> 1.583069,
> 1.963791,
> 1.773041,
> 1.890228,
> 1.793457
> ]
> ]
> },
plan1 (no change)
> ceph osd crush weight-set reweight-compat 24 1.756317
> ceph osd crush weight-set reweight-compat 25 1.613647
> ceph osd crush weight-set reweight-compat 26 1.733200
> ceph osd crush weight-set reweight-compat 27 1.735825
> ceph osd crush weight-set reweight-compat 28 1.961304
> ceph osd crush weight-set reweight-compat 29 1.583069
> ceph osd crush weight-set reweight-compat 30 1.963791
> ceph osd crush weight-set reweight-compat 31 1.773041
> ceph osd crush weight-set reweight-compat 32 1.890228
> ceph osd crush weight-set reweight-compat 33 1.793457
plan2 (change)
> ceph osd crush weight-set reweight-compat 24 1.742185
> ceph osd crush weight-set reweight-compat 25 1.608330
> ceph osd crush weight-set reweight-compat 26 1.753393
> ceph osd crush weight-set reweight-compat 27 1.713531
> ceph osd crush weight-set reweight-compat 28 1.964446
> ceph osd crush weight-set reweight-compat 29 1.629001
> ceph osd crush weight-set reweight-compat 30 1.961968
> ceph osd crush weight-set reweight-compat 31 1.738253
> ceph osd crush weight-set reweight-compat 32 1.884098
> ceph osd crush weight-set reweight-compat 33 1.779180
Hopefully this will be helpful for someone else who overlooks this in the -h
output.
Reed
> On Aug 1, 2018, at 6:05 PM, Reed Dier <[email protected]> wrote:
>
> Hi Cephers,
>
> I’m starting to play with the Ceph Balancer plugin after moving to straw2 and
> running into something I’m surprised I haven’t seen posted here.
>
> My cluster has two crush roots, one for HDD, one for SSD.
>
> Right now, HDD’s are a single pool to themselves, SSD’s are a single pool to
> themselves.
>
> Using Ceph Balancer Eval, I can see the eval score for the hdd’s (worse), and
> the ssd’s (better), and the blended score of the cluster overall.
> pool “hdd" score 0.012529 (lower is better)
> pool “ssd" score 0.004654 (lower is better)
> current cluster score 0.008484 (lower is better)
>
> My problem is that I need to get my hdd’s better, and stop touching my ssd's,
> because shuffling data wear’s the ssd's unnecessarily, and it has actually
> gotten the distribution worse over time. https://imgur.com/RVh0jfH
> <https://imgur.com/RVh0jfH>
> You can see that between 06:00 and 09:00 on the second day in the graph that
> the spread was very tight, and then it expanded back.
>
> So my question is, how can I run the balancer on just my hdd’s without
> touching my ssd’s?
>
> I removed about 15% of the PG’s living on the HDD’s because they were empty.
> I also have two tiers of HDD’s 8TB’s and 2TB’s, but they are roughly equally
> weighted in crush at the chassis level where my failure domains are
> configured.
> Hopefully this abbreviated ceph osd tree displays the hierarchy. Multipliers
> for that bucket on right.
>> ID CLASS WEIGHT TYPE NAME
>> -1 218.49353 root default.hdd
>> -10 218.49353 rack default.rack-hdd
>> -70 43.66553 chassis hdd-2tb-chassis1 *1
>> -67 43.66553 host hdd-2tb-24-1 *1
>> 74 hdd 1.81940 osd.74 *24
>> -55 43.70700 chassis hdd-8tb-chassis1 *4
>> -2 21.85350 host hdd-8tb-3-1
>> 0 hdd 7.28450 osd.0 *3
>> -3 21.85350 host hdd-8tb-3-1
>> 1 hdd 7.28450 osd.1 *3
>
>
> I assume this doesn’t complicate totally, but figured I would mention it, as
> I assume it is more difficult to equally distribute across OSD’s that are 4:1
> size delta.
>
>
> If I create a plan plan1 with ceph balancer optimize plan1,
> then do a show plan1, I see an entry:
> ceph osd crush weight-set reweight-compat $OSD $ArbitraryNumberNearOsdSize
>
> Could I then copy this output, remove entries for SSD OSD’s and then run the
> ceph osd crush weight-set reweight-compat commands in a script?
>
> I and my SSD’s appreciate any insight.
>
> Thanks,
>
> Reed
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
