Sage says a whole bunch of fixes for this have gone in since both then and 12.1.2. We should be pushing out a final 12.1.3 today for people to test on; can you try that and report back once it's out? -Greg
On Thu, Aug 10, 2017 at 8:32 AM Sean Purdy <s.pu...@cv-library.co.uk> wrote: > Luminous 12.1.1 rc > > > Our OSD osd.8 failed. So we removed that. > > > We added a new disk and did: > > $ ceph-deploy osd create --dmcrypt --bluestore store02:/dev/sdd > > That worked, created osd.18, OSD has data. > > However, mgr output at http://localhost:7000/servers showed > osd.18 under a blank hostname and not e.g. on the node we attached it to. > But it is working. "ceph osd tree" looks OK > > > The problem I see is: > When I do "ceph osd crush tree" I see the items list under the > name:default~hdd tree: > > device_class:hdd > name:store02~hdd > type:host > > but my new drive is missing under this name - there are 5 OSDs, not 6. > > > *However*, if I look further down under the name:default tree > > device_class:"" > name:store02 > type:host > > I see all devices I am expecting, including osd.18 > > > Is this something to worry about? Or is there something needs fixing? > Health is warning for scrubbing reasons. > > > Output of related commands below. > > > Thanks for any help, > > Sean Purdy > > > $ sudo ceph osd tree > ID CLASS WEIGHT TYPE NAME UP/DOWN REWEIGHT PRI-AFF > -1 32.73651 root default > -3 10.91217 host store01 > 0 hdd 1.81870 osd.0 up 1.00000 1.00000 > 5 hdd 1.81870 osd.5 up 1.00000 1.00000 > 6 hdd 1.81870 osd.6 up 1.00000 1.00000 > 9 hdd 1.81870 osd.9 up 1.00000 1.00000 > 12 hdd 1.81870 osd.12 up 1.00000 1.00000 > 15 hdd 1.81870 osd.15 up 1.00000 1.00000 > -5 10.91217 host store02 > 1 hdd 1.81870 osd.1 up 1.00000 1.00000 > 7 hdd 1.81870 osd.7 up 1.00000 1.00000 > 10 hdd 1.81870 osd.10 up 1.00000 1.00000 > 13 hdd 1.81870 osd.13 up 1.00000 1.00000 > 16 hdd 1.81870 osd.16 up 1.00000 1.00000 > 18 hdd 1.81870 osd.18 up 1.00000 1.00000 > -7 10.91217 host store03 > 2 hdd 1.81870 osd.2 up 1.00000 1.00000 > 3 hdd 1.81870 osd.3 up 1.00000 1.00000 > 4 hdd 1.81870 osd.4 up 1.00000 1.00000 > 11 hdd 1.81870 osd.11 up 1.00000 1.00000 > 14 hdd 1.81870 osd.14 up 1.00000 1.00000 > 17 hdd 1.81870 osd.17 up 1.00000 1.00000 > > > $ sudo ceph osd crush tree > [ > { > "id": -8, > "device_class": "hdd", > "name": "default~hdd", > "type": "root", > "type_id": 10, > "items": [ > { > "id": -2, > "device_class": "hdd", > "name": "store01~hdd", > "type": "host", > "type_id": 1, > "items": [ > { > "id": 0, > "device_class": "hdd", > "name": "osd.0", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 5, > "device_class": "hdd", > "name": "osd.5", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 6, > "device_class": "hdd", > "name": "osd.6", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 9, > "device_class": "hdd", > "name": "osd.9", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 12, > "device_class": "hdd", > "name": "osd.12", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 15, > "device_class": "hdd", > "name": "osd.15", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > } > ] > }, > { > "id": -4, > "device_class": "hdd", > "name": "store02~hdd", > "type": "host", > "type_id": 1, > "items": [ > { > "id": 1, > "device_class": "hdd", > "name": "osd.1", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 7, > "device_class": "hdd", > "name": "osd.7", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 10, > "device_class": "hdd", > "name": "osd.10", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 13, > "device_class": "hdd", > "name": "osd.13", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 16, > "device_class": "hdd", > "name": "osd.16", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > } > ] > }, > { > "id": -6, > "device_class": "hdd", > "name": "store03~hdd", > "type": "host", > "type_id": 1, > "items": [ > { > "id": 2, > "device_class": "hdd", > "name": "osd.2", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 3, > "device_class": "hdd", > "name": "osd.3", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 4, > "device_class": "hdd", > "name": "osd.4", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 11, > "device_class": "hdd", > "name": "osd.11", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 14, > "device_class": "hdd", > "name": "osd.14", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 17, > "device_class": "hdd", > "name": "osd.17", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > } > ] > } > ] > }, > { > "id": -1, > "device_class": "", > "name": "default", > "type": "root", > "type_id": 10, > "items": [ > { > "id": -3, > "device_class": "", > "name": "store01", > "type": "host", > "type_id": 1, > "items": [ > { > "id": 0, > "device_class": "hdd", > "name": "osd.0", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 5, > "device_class": "hdd", > "name": "osd.5", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 6, > "device_class": "hdd", > "name": "osd.6", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 9, > "device_class": "hdd", > "name": "osd.9", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 12, > "device_class": "hdd", > "name": "osd.12", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 15, > "device_class": "hdd", > "name": "osd.15", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > } > ] > }, > { > "id": -5, > "device_class": "", > "name": "store02", > "type": "host", > "type_id": 1, > "items": [ > { > "id": 1, > "device_class": "hdd", > "name": "osd.1", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 7, > "device_class": "hdd", > "name": "osd.7", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 10, > "device_class": "hdd", > "name": "osd.10", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 13, > "device_class": "hdd", > "name": "osd.13", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 16, > "device_class": "hdd", > "name": "osd.16", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 18, > "device_class": "hdd", > "name": "osd.18", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > } > ] > }, > { > "id": -7, > "device_class": "", > "name": "store03", > "type": "host", > "type_id": 1, > "items": [ > { > "id": 2, > "device_class": "hdd", > "name": "osd.2", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 3, > "device_class": "hdd", > "name": "osd.3", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 4, > "device_class": "hdd", > "name": "osd.4", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 11, > "device_class": "hdd", > "name": "osd.11", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 14, > "device_class": "hdd", > "name": "osd.14", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > }, > { > "id": 17, > "device_class": "hdd", > "name": "osd.17", > "type": "osd", > "type_id": 0, > "crush_weight": 1.818695, > "depth": 2 > } > ] > } > ] > } > ] > > > $ sudo ceph status > cluster: > id: a8c34694-a172-4418-a7dd-dd8a642eb545 > health: HEALTH_WARN > 32 pgs not deep-scrubbed for 86400 > 174 pgs not scrubbed for 86400 > > services: > mon: 3 daemons, quorum store01,store02,store03 > mgr: store01(active) > osd: 18 osds: 18 up, 18 in > rgw: 3daemonss active > > data: > pools: 6 pools, 672 pgs > objects: 4018k objects, 303 GB > usage: 1482 GB used, 32040 GB / 33523 GB avail > pgs: 672 active+clean > > io: > client: 72099 B/s rd, 542 kB/s wr, 70 op/s rd, 102 op/s wr > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
