So, currently this is what /var/lib/ceph/osd/ceph-60 shows. Is it not correct? I don't know what I should expect to see.
root@osd1:~# ls -l /var/lib/ceph/osd/ceph-60 total 86252 -rw-r--r-- 1 ceph ceph 384 Nov 2 16:20 activate.monmap -rw-r--r-- 1 ceph ceph 10737418240 Nov 5 16:32 block lrwxrwxrwx 1 ceph ceph 14 Nov 2 16:20 block.db -> /dev/ssd0/db60 -rw-r--r-- 1 ceph ceph 2 Nov 2 16:20 bluefs -rw-r--r-- 1 ceph ceph 37 Nov 2 16:20 ceph_fsid -rw-r--r-- 1 ceph ceph 37 Nov 2 16:20 fsid -rw------- 1 ceph ceph 57 Nov 2 16:20 keyring -rw-r--r-- 1 ceph ceph 8 Nov 2 16:20 kv_backend -rw-r--r-- 1 ceph ceph 21 Nov 2 16:20 magic -rw-r--r-- 1 ceph ceph 4 Nov 2 16:20 mkfs_done -rw-r--r-- 1 ceph ceph 41 Nov 2 16:20 osd_key -rw-r--r-- 1 ceph ceph 6 Nov 2 16:20 ready -rw-r--r-- 1 ceph ceph 10 Nov 2 16:20 type -rw-r--r-- 1 ceph ceph 3 Nov 2 16:20 whoami The disk I am using for this osd (osd.60) is a 3.7TB hdd. lsblk shows sdh 8:112 0 3.7T 0 disk └─hdd60-data60 252:1 0 3.7T 0 lvm and "ceph osd tree" shows 60 hdd 3.63689 osd.60 up 1.00000 1.00000 On Mon, Nov 5, 2018 at 11:23 PM, Hector Martin <hec...@marcansoft.com> wrote: > On 11/6/18 6:03 AM, Hayashida, Mami wrote: > > WOW. With you two guiding me through every step, the 10 OSDs in > > question are now added back to the cluster as Bluestore disks!!! Here > > are my responses to the last email from Hector: > > > > 1. I first checked the permissions and they looked like this > > > > root@osd1:/var/lib/ceph/osd/ceph-60# ls -l > > total 56 > > -rw-r--r-- 1 ceph ceph 384 Nov 2 16:20 activate.monmap > > -rw-r--r-- 1 ceph ceph 10737418240 Nov 2 16:20 block > > lrwxrwxrwx 1 ceph ceph 14 Nov 2 16:20 block.db -> > /dev/ssd0/db60 > > Wait. > > That's not right, is it? That's a 10GB raw file being used as a > BlueStore block device. It should be a symlink to an LVM volume, not a > file. > > Can you check the `block` files/symlinks again for all the OSDs and also > what is mounted on the OSD directories? It should be tmpfs directories > with symlinks to block devices. I'm not sure what happened there. > > -- > Hector Martin (hec...@marcansoft.com) > Public Key: https://mrcn.st/pub > -- *Mami Hayashida* *Research Computing Associate* Research Computing Infrastructure University of Kentucky Information Technology Services 301 Rose Street | 102 James F. Hardymon Building Lexington, KY 40506-0495 mami.hayash...@uky.edu (859)323-7521
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
