You're mixing up non-cephadm with cephadm cluster commands. Don't use
ceph-volume directly. Try the 'ceph cephadm osd activate <host>'
command instead, then inspect osd logs and ceph-volume.log in case it
fails.
Zitat von Jacek Rużyczka <[email protected]>:
Hi,
I must have followed the wrong instructions (as often seems to be the case
when it comes to Ceph). I gave Ceph another try with --fsid, and then it
worked.
A missing fsid file cost me an hour, but the OSD subfolder seems to be OK
now:
mixtile@blade3n1:~$ sudo ls -al
/var/lib/ceph/8aad3073-39a1-11f1-bf6e-f2704a1efa
9b/osd.2
total 16
drwxr-xr-x 2 ceph ceph 4096 Jun 1 16:50 .
drwx------ 14 167 167 4096 Jun 1 13:47 ..
lrwxrwxrwx 1 ceph ceph 93 Jun 1 16:44 block ->
/dev/ceph-81bc1761-e8f6-446c-
96f3-eb1b8f92628b/osd-block-9f7fd40d-0698-40b9-8718-62942b03e263
-rw-r--r-- 1 ceph ceph 37 Jun 1 16:50 fsid
In the meantime, I've managed to issue this command: sudo ceph-volume lvm
activate --all
There's one problem though: I get the feedback that ceph-osd@2 is working,
but sudo systemctl status ceph-osd@2
says that the daemon failed to start:
Jun 01 18:57:43 blade3n1 ceph-osd[317772]: 2026-06-01T18:57:43.403+0200
ffffb584
9040 -1 osd.2 7314 init authentication failed: (13) Permission denied
What puzzles me: Why is the OSD daemon started as a normal service and not
as a Docker container?
Am So., 31. Mai 2026 um 21:27 Uhr schrieb Eugen Block via ceph-users <
[email protected]>:
I don't see the '--fsid' flag in your command, it needs to be part of
the bootstrap command. So 'cephadm bootrap --mon-ip IP --fsid FSID'
and so on.
4 OSDs istn't that much, it should be doable in a reasonable amount of
time. It took me around two hours to bring my test cluster with 3 OSDs
back up, I think.
> /var/lib/ceph/osd/ceph-2
This is the view from within the cephadm shell, the directory on the
filesystem is :
/var/lib/ceph/{FSID}/osd.2
which is then mapped into the container.
Zitat von Jacek Rużyczka <[email protected]>:
>>
>> No, the data is not inaccessible until you wipe the OSDs.
>
>
> Ah, that's good to know that at least the data are still there.
>
> How many OSDs are we talking about here?
>
>
> I've got 4 OSDs, which hold 1.3 TB of data in total. I do have a backup
of
> them on an external HDD, which spit out a handful of warnings during the
> last restore, so destroying the previous SSD, creating new ones, and
trying
> another restore are things I'd like to avoid.
>
> And regarding the fsid, you can specify it in the bootstrap command,
>> you won’t be able to change it afterwards.
>
>
> This is exactly what I tried to do:
>
> sudo cephadm bootstrap --mon-ip $MYIP --config /etc/ceph/ceph.conf~~
> --allow-overwrite
>
> And /etc/ceph/ceph.conf~~ contains the old FSID:
>
> # minimal ceph.conf for 8aad3073-39a1-11f1-bf6e-f2704a1efa9b
> [global]
> fsid = 8aad3073-39a1-11f1-bf6e-f2704a1efa9b
> mon_host = [v2:10.20.0.11:3300/0,v1:10.20.0.11:6789/0] [v2:
> 10.20.0.12:3300/0,v1:10.20.0.12:6789/0] [v2:
> 10.20.0.13:3300/0,v1:10.20.0.13:6789/0] [v2:
> 10.20.0.14:3300/0,v1:10.20.0.14:6789/0]
>
> Nevertheless, cephadm ignored the old settings and gave the new cluster
a
> generated ID. So I've gotta tear down the cluster and start again? What
> about the key pairs stored in /etc/ceph?
>
> You can simply create the OSD directories and create a symbolic link to
>> the block device. Cephadm simply maps it then.
>
>
> I've now recreated the dir /var/lib/ceph/osd/ceph-2 and have to symlink
> /dev/ceph-81bc1761-e8f6-446c-96f3-eb1b8f92628b/o
> sd-block-9f7fd40d-0698-40b9-8718-62942b03e263 there? Or should ceph-2 be
> the name of the symlink?
>
> I recommend to hire a professional to help you out, just to be on the
safe
>> side.
>
>
> I've got a Ceph expert in our hackspace, but I'm not sure whether he can
> help me.
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
