For a moment it de-list removed OSD's and after sometime it again comes up in ceph osd tree listing.
On Fri, Jul 31, 2015 at 12:45 PM, Mallikarjun Biradar <mallikarjuna.bira...@gmail.com> wrote: > Hi, > > I had 27 OSD's in my cluster. I removed two of the OSD from (osd.20) > host-3 & (osd.22) host-6. > > user@host-1:~$ sudo ceph osd tree > ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY > -1 184.67990 root default > -7 82.07996 chassis chassis2 > -4 41.03998 host host-3 > 8 6.84000 osd.8 up 1.00000 1.00000 > 9 6.84000 osd.9 up 1.00000 1.00000 > 10 6.84000 osd.10 up 1.00000 1.00000 > 11 6.84000 osd.11 up 1.00000 1.00000 > 20 6.84000 osd.20 up 1.00000 1.00000 > 21 6.84000 osd.21 up 1.00000 1.00000 > -5 41.03998 host host-6 > 12 6.84000 osd.12 up 1.00000 1.00000 > 13 6.84000 osd.13 up 1.00000 1.00000 > 14 6.84000 osd.14 up 1.00000 1.00000 > 15 6.84000 osd.15 up 1.00000 1.00000 > 22 6.84000 osd.22 up 1.00000 1.00000 > 23 6.84000 osd.23 up 1.00000 1.00000 > -6 102.59995 chassis chassis1 > -2 47.87997 host host-1 > 0 6.84000 osd.0 up 1.00000 1.00000 > 1 6.84000 osd.1 up 1.00000 1.00000 > 2 6.84000 osd.2 up 1.00000 1.00000 > 3 6.84000 osd.3 up 1.00000 1.00000 > 16 6.84000 osd.16 up 1.00000 1.00000 > 17 6.84000 osd.17 up 1.00000 1.00000 > 24 6.84000 osd.24 up 1.00000 1.00000 > -3 54.71997 host host-2 > 4 6.84000 osd.4 up 1.00000 1.00000 > 5 6.84000 osd.5 up 1.00000 1.00000 > 6 6.84000 osd.6 up 1.00000 1.00000 > 7 6.84000 osd.7 up 1.00000 1.00000 > 18 6.84000 osd.18 up 1.00000 1.00000 > 19 6.84000 osd.19 up 1.00000 1.00000 > 25 6.84000 osd.25 up 1.00000 1.00000 > 26 6.84000 osd.26 up 1.00000 1.00000 > user@host-1:~$ > > Steps used to remove OSD: > user@host-1:~$ ceph auth del osd.20; ceph osd crush rm osd.20; ceph > osd down osd.20; ceph osd rm osd.20 > updated > removed item id 20 name 'osd.20' from crush map > marked down osd.22. > removed osd.22 > > Removed both of OSD's osd.20 & osd.22 > > But, even after removing them, ceph osd tree is listing deleted OSD's > & ceph -s reporting total number of OSD's as 27. > > user@host-1:~$ sudo ceph osd tree > ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY > -1 184.67990 root default > -7 82.07996 chassis chassis2 > -4 41.03998 host host-3 > 8 6.84000 osd.8 up 1.00000 1.00000 > 9 6.84000 osd.9 up 1.00000 1.00000 > 10 6.84000 osd.10 up 1.00000 1.00000 > 11 6.84000 osd.11 up 1.00000 1.00000 > 21 6.84000 osd.21 up 1.00000 1.00000 > -5 41.03998 host host-6 > 12 6.84000 osd.12 up 1.00000 1.00000 > 13 6.84000 osd.13 up 1.00000 1.00000 > 14 6.84000 osd.14 up 1.00000 1.00000 > 15 6.84000 osd.15 up 1.00000 1.00000 > 23 6.84000 osd.23 up 1.00000 1.00000 > -6 102.59995 chassis chassis1 > -2 47.87997 host host-1 > 0 6.84000 osd.0 up 1.00000 1.00000 > 1 6.84000 osd.1 up 1.00000 1.00000 > 2 6.84000 osd.2 up 1.00000 1.00000 > 3 6.84000 osd.3 up 1.00000 1.00000 > 16 6.84000 osd.16 up 1.00000 1.00000 > 17 6.84000 osd.17 up 1.00000 1.00000 > 24 6.84000 osd.24 up 1.00000 1.00000 > -3 54.71997 host host-2 > 4 6.84000 osd.4 up 1.00000 1.00000 > 5 6.84000 osd.5 up 1.00000 1.00000 > 6 6.84000 osd.6 up 1.00000 1.00000 > 7 6.84000 osd.7 up 1.00000 1.00000 > 18 6.84000 osd.18 up 1.00000 1.00000 > 19 6.84000 osd.19 up 1.00000 1.00000 > 25 6.84000 osd.25 up 1.00000 1.00000 > 26 6.84000 osd.26 up 1.00000 1.00000 > 20 0 osd.20 up 0 > 1.00000 > 22 0 osd.22 up 0 > 1.00000 > user@host-1:~$ > > Please let me know how to remove the OSD in this case. > > -Thanks & Regards, > Mallikarjun Biradar _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
