I commented out those lines and, yes, I was able to restart the system and
all the Filestore OSDs are now running. But when I cannot start converted
Bluestore OSDs (service). When I look up the log for osd.60, this is what
I see:
2018-11-05 12:47:00.756794 7f1f2775ae00 0 set uid:gid to 64045:64045
(ceph:ceph)
2018-11-05 12:47:00.756821 7f1f2775ae00 0 ceph version 12.2.9
(9e300932ef8a8916fb3fda78c58691a6ab0f4217) luminous (stable), process
ceph-osd, pid 33706
2018-11-05 12:47:00.776554 7f1f2775ae00 0 pidfile_write: ignore empty
--pid-file
2018-11-05 12:47:00.788716 7f1f2775ae00 0 load: jerasure load: lrc load:
isa
2018-11-05 12:47:00.788803 7f1f2775ae00 1 bdev create path
/var/lib/ceph/osd/ceph-60/block type kernel
2018-11-05 12:47:00.788818 7f1f2775ae00 1 bdev(0x564350f4ad80
/var/lib/ceph/osd/ceph-60/block) open path /var/lib/ceph/osd/ceph-60/block
2018-11-05 12:47:00.789179 7f1f2775ae00 1 bdev(0x564350f4ad80
/var/lib/ceph/osd/ceph-60/block) open size 10737418240 (0x280000000, 10GiB)
block_size 4096 (4KiB) rotational
2018-11-05 12:47:00.789257 7f1f2775ae00 1
bluestore(/var/lib/ceph/osd/ceph-60) _set_cache_sizes cache_size 1073741824
meta 0.4 kv 0.4 data 0.2
2018-11-05 12:47:00.789286 7f1f2775ae00 1 bdev(0x564350f4ad80
/var/lib/ceph/osd/ceph-60/block) close
2018-11-05 12:47:01.075002 7f1f2775ae00 1
bluestore(/var/lib/ceph/osd/ceph-60) _mount path /var/lib/ceph/osd/ceph-60
2018-11-05 12:47:01.075069 7f1f2775ae00 1 bdev create path
/var/lib/ceph/osd/ceph-60/block type kernel
2018-11-05 12:47:01.075078 7f1f2775ae00 1 bdev(0x564350f4afc0
/var/lib/ceph/osd/ceph-60/block) open path /var/lib/ceph/osd/ceph-60/block
2018-11-05 12:47:01.075391 7f1f2775ae00 1 bdev(0x564350f4afc0
/var/lib/ceph/osd/ceph-60/block) open size 10737418240 (0x280000000, 10GiB)
block_size 4096 (4KiB) rotational
2018-11-05 12:47:01.075450 7f1f2775ae00 1
bluestore(/var/lib/ceph/osd/ceph-60) _set_cache_sizes cache_size 1073741824
meta 0.4 kv 0.4 data 0.2
2018-11-05 12:47:01.075536 7f1f2775ae00 1 bdev create path
/var/lib/ceph/osd/ceph-60/block.db type kernel
2018-11-05 12:47:01.075544 7f1f2775ae00 1 bdev(0x564350f4b200
/var/lib/ceph/osd/ceph-60/block.db) open path
/var/lib/ceph/osd/ceph-60/block.db
2018-11-05 12:47:01.075555 7f1f2775ae00 -1 bdev(0x564350f4b200
/var/lib/ceph/osd/ceph-60/block.db) open open got: (13) Permission denied
2018-11-05 12:47:01.075573 7f1f2775ae00 -1
bluestore(/var/lib/ceph/osd/ceph-60) _open_db add block
device(/var/lib/ceph/osd/ceph-60/block.db) returned: (13) Permission denied
2018-11-05 12:47:01.075589 7f1f2775ae00 1 bdev(0x564350f4afc0
/var/lib/ceph/osd/ceph-60/block) close
2018-11-05 12:47:01.346356 7f1f2775ae00 -1 osd.60 0 OSD:init: unable to
mount object store
2018-11-05 12:47:01.346378 7f1f2775ae00 -1 ** ERROR: osd init failed: (13)
Permission denied
On Mon, Nov 5, 2018 at 12:34 PM, Hector Martin <hec...@marcansoft.com>
wrote:
> On 11/6/18 2:01 AM, Hayashida, Mami wrote:
> > I did find in /etc/fstab entries like this for those 10 disks
> >
> > /dev/sdh1 /var/lib/ceph/osd/ceph-60 xfs noatime,nodiratime 0 0
> >
> > Should I comment all 10 of them out (for osd.{60-69}) and try rebooting
> > again?
>
> Yes. Anything that references any of the old partitions that don't exist
> (/dev/sdh1 etc) should be removed. The disks are now full-disk LVM PVs
> and should have no partitions.
>
> --
> Hector Martin (hec...@marcansoft.com)
> Public Key: https://mrcn.st/pub
>
--
*Mami Hayashida*
*Research Computing Associate*
Research Computing Infrastructure
University of Kentucky Information Technology Services
301 Rose Street | 102 James F. Hardymon Building
Lexington, KY 40506-0495
mami.hayash...@uky.edu
(859)323-7521
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
