Am 2025-05-16 18:01, schrieb Anthony D'Atri:
I wouldn’t think blkdiscard would necessarily fully clean. I would try
sgdisk —zap-all or Ceph-volume lvm zap
I gave this a try in addition to a reboot but no changes. Still not
created a bluestore osd as intended.
I guess this is the culprit:
2025-05-16T16:28:14.865+0000 7be7486fb880 -1
bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to
decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void
bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)
decode past end of struct encoding: Malformed input [buffer:3]
Maybe someone has a hint?
Yours
--martin
------------------------------------------------------------------------------
# ceph-volume lvm prepare --bluestore --data vgsdd/sdd1
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name
client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
-i - osd new a5ae5940-26f2-44c8-bf16-bfdfa6de20ba
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-1
--> Executable selinuxenabled not in PATH:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Running command: /usr/bin/chown -h ceph:ceph /dev/vgsdd/sdd1
Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7
Running command: /usr/bin/ln -s /dev/vgsdd/sdd1
/var/lib/ceph/osd/ceph-1/block
Running command: /usr/bin/ceph --cluster ceph --name
client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap
stderr: got monmap epoch 20
--> Creating keyring file for osd.1
Running command: /usr/bin/chown -R ceph:ceph
/var/lib/ceph/osd/ceph-1/keyring
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/
Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore
bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap
--keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid
a5ae5940-26f2-44c8-bf16-bfdfa6de20ba --setuser ceph --setgroup ceph
stderr: 2025-05-16T16:28:14.865+0000 7be7486fb880 -1
bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to
decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void
bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)
decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2025-05-16T16:28:14.865+0000 7be7486fb880 -1
bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to
decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void
bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)
decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2025-05-16T16:28:14.866+0000 7be7486fb880 -1
bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to
decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void
bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)
decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2025-05-16T16:28:14.867+0000 7be7486fb880 -1
bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to
decode label /var/lib/ceph/osd/ceph-1//block at offset 102: void
bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)
decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2025-05-16T16:28:14.867+0000 7be7486fb880 -1
bluestore(/var/lib/ceph/osd/ceph-1//block) No valid bdev label found
stderr: 2025-05-16T16:28:15.127+0000 7be7486fb880 -1
bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid
--> ceph-volume lvm prepare successful for: vgsdd/sdd1
root@pve-03 (pve-03.t3):~# ceph-volume lvm activate --all
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
--> OSD ID 0 FSID a764e57e-ad3b-4823-8849-454f133526f6 process is
active. Skipping activation
--> Activating OSD ID 1 FSID a5ae5940-26f2-44c8-bf16-bfdfa6de20ba
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1
Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph
prime-osd-dir --dev /dev/vgsdd/sdd1 --path /var/lib/ceph/osd/ceph-1
--no-mon-config
Running command: /usr/bin/ln -snf /dev/vgsdd/sdd1
/var/lib/ceph/osd/ceph-1/block
Running command: /usr/bin/chown -h ceph:ceph
/var/lib/ceph/osd/ceph-1/block
Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1
Running command: /usr/bin/systemctl enable
ceph-volume@lvm-1-a5ae5940-26f2-44c8-bf16-bfdfa6de20ba
stderr: Created symlink
/etc/systemd/system/multi-user.target.wants/ceph-volume@lvm-1-a5ae5940-26f2-44c8-bf16-bfdfa6de20ba.service
→ /lib/systemd/system/ceph-volume@.service.
Running command: /usr/bin/systemctl enable --runtime ceph-osd@1
stderr: Created symlink
/run/systemd/system/ceph-osd.target.wants/ceph-osd@1.service →
/lib/systemd/system/ceph-osd@.service.
Running command: /usr/bin/systemctl start ceph-osd@1
--> ceph-volume lvm activate successful for osd ID: 1
ceph-volume-systemd.log has:
[2025-05-16 16:23:43,600][ceph_volume.process][INFO ] stderr -->
RuntimeError: could not find osd.1 with osd_fsid
a2a1cb42-006f-4659-a461-bb0ffbf190ff
ps:
ceph 11572 0.5 0.0 689688 83000 ? Ssl 16:29 0:02
/usr/bin/ceph-osd -f --cluster ceph --id 1 --setuser ceph --setgroup
ceph
# systemctl status ceph-osd@1.service
● ceph-osd@1.service - Ceph object storage daemon osd.1
Loaded: loaded (/lib/systemd/system/ceph-osd@.service;
enabled-runtime; preset: enabled)
Drop-In: /usr/lib/systemd/system/ceph-osd@.service.d
└─ceph-after-pve-cluster.conf
Active: active (running) since Fri 2025-05-16 16:29:11 UTC; 7min
ago
Process: 11567 ExecStartPre=/usr/libexec/ceph/ceph-osd-prestart.sh
--cluster ${CLUSTER} --id 1 (code=exited, status=0/SUCCESS)
Main PID: 11572 (ceph-osd)
Tasks: 68
Memory: 49.4M
CPU: 2.239s
CGroup: /system.slice/system-ceph\x2dosd.slice/ceph-osd@1.service
└─11572 /usr/bin/ceph-osd -f --cluster ceph --id 1
--setuser ceph --setgroup ceph
Mai 16 16:29:11 pve-03 systemd[1]: Starting ceph-osd@1.service - Ceph
object storage daemon osd.1...
Mai 16 16:29:11 pve-03 systemd[1]: Started ceph-osd@1.service - Ceph
object storage daemon osd.1.
Mai 16 16:29:15 pve-03 ceph-osd[11572]: 2025-05-16T16:29:15.066+0000
7b3558e37880 -1 osd.1 0 log_to_monitors true
Mai 16 16:29:16 pve-03 ceph-osd[11572]: 2025-05-16T16:29:16.351+0000
7b3554dd76c0 -1 osd.1 0 waiting for initial osdmap
Mai 16 16:29:16 pve-03 ceph-osd[11572]: 2025-05-16T16:29:16.352+0000
7b354c6166c0 -1 osd.1 0 failed to load OSD map for epoch 32827, got 0
bytes
Mai 16 16:29:16 pve-03 ceph-osd[11572]: 2025-05-16T16:29:16.657+0000
7b3548b646c0 -1 osd.1 33967 osdmap NOUP flag is set, waiting for it to
clear
Mai 16 16:29:17 pve-03 ceph-osd[11572]: 2025-05-16T16:29:17.354+0000
7b3548b646c0 -1 osd.1 33968 osdmap NOUP flag is set, waiting for it to
clear
Mai 16 16:29:18 pve-03 ceph-osd[11572]: 2025-05-16T16:29:18.362+0000
7b3548b646c0 -1 osd.1 33969 osdmap NOUP flag is set, waiting for it to
clear
--
Martin Konold - Prokurist, CTO
KONSEC GmbH - make things real
Amtsgericht Stuttgart, HRB 23690
Geschäftsführer: Andreas Mack
Im Köller 3, 70794 Filderstadt, Germany
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
