That’s really surprising, did you also already have to replace drives
and they were redeployed as expected?
I must admit, it’s been some time since I last tried something similar
(maybe pacific), and when it failed I decided to do it differently.
Which version is this?
Zitat von Stephan Hohn <[email protected]>:
In our deployments it works like this.
It grabs what you get from "ceph orch ls" from top to bottom.
That's why we use the numbering (osd_1... osd_2... osd_3...).
Also db_slots work quite well.
Am Di., 30. Sept. 2025 um 15:57 Uhr schrieb Eugen Block <[email protected]>:
That's not going to work as you expect: if the same host is applicable
to multiple osd specs, only one of the specs will be applied (I think
it's the last one in the list of 'ceph orch ls osd --export').
I would also recommend to not use device paths, those can change after
a reboot (although cephadm works with LVM and labels).
The config option "db_slots" has never worked reliably, it's mentioned
in the docs somewhere, so it most likely won't work either as you
expect.
If you can distinguish your OSDs using models, sizes or some other
parameter, you could use several osd specs. If you can't, you might
need to create the daemons manually like this:
ceph01:~ # ceph orch daemon add osd
ceph01:data_devices=/dev/vdb,/dev/vdc,/dev/vdd,db_devices=/dev/vde,block_db_size=64G
(I'm not sure about that last parameter, writing this from the top of my
head)
But note that these OSDs will be displayed as "unmanaged" in 'ceph
orch ls osd' output.
Zitat von "GLE, Vivien" <[email protected]>:
> Hi,
>
>
> For testing purpose we need to deploy :
>
>
> - 1 pool of 6 SSD OSD
>
> - 1 pool of 6 HDD OSD
>
> - 1 pool of 6 HDD OSD with 2 SSD for DB+WAL
>
>
> I tried to orch apply this yaml but it doesn't work as expected
>
>
> Osd part of the yaml ->
>
>
> service_type: osd
> service_id: osd_spec
> placement:
> hosts:
> - host1
> - host2
> - host3
> - host4
> - host5
> - host6
> data_devices:
> paths:
> - /dev/sda
> - /dev/sdc
> spec:
> data_devices:
> all: true
> filter_logic: AND
> objectstore: bluestore
> ---
> service_type: osd
> service_id: osd_spec_wall
> placement:
> hosts:
> - host1
> - host2
> - host3
> - host4
> - host5
> - host6
>
> spec:
> data_devices:
> paths:
> - /dev/sdf
> db_devices:
> paths:
> - /dev/sde
> limit: 2
> db_slots: 3
>
>
>
> Only 1 db on /dev/sde from host1 has been created and this OSD
> showed up as half full at his creation:
>
> ceph osd df | grep 25
>
> 25 hdd 0.63669 1.00000 652 GiB 373 GiB 1.5 MiB 1 KiB 38
> MiB 279 GiB 57.15 35.93 0 up
>
> ceph-volume lvm list
>
> ====== osd.25 ======
>
> [block]
>
/dev/ceph-23d9297a-d0e1-47be-8650-5c8ccae4fe0e/osd-block-2f009760-fc2b-46d5-984d-e8200dfd9d9d
>
> block device
>
/dev/ceph-23d9297a-d0e1-47be-8650-5c8ccae4fe0e/osd-block-2f009760-fc2b-46d5-984d-e8200dfd9d9d
> block uuid vqJGf8-V5g0-S1cA-BAcN-Qm9D-9VTx-xsc8wk
> cephx lockbox secret
> cluster fsid id
> cluster name ceph
> crush device class
> db device
>
/dev/ceph-9c39b87c-a39c-413f-b1ef-07881195fcb8/osd-db-feee5095-b5e7-47a0-ae87-8e5039512661
> db uuid Rtj5KA-Qxjk-IFmY-3ffQ-gSAu-Snte-uku8HC
> encrypted 0
> osd fsid 2f009760-fc2b-46d5-984d-e8200dfd9d9d
> osd id 25
> osdspec affinity osd_spec_wall
> type block
> vdo 0
> with tpm 0
> devices /dev/sdf
>
> [db]
>
/dev/ceph-9c39b87c-a39c-413f-b1ef-07881195fcb8/osd-db-feee5095-b5e7-47a0-ae87-8e5039512661
>
> block device
>
/dev/ceph-23d9297a-d0e1-47be-8650-5c8ccae4fe0e/osd-block-2f009760-fc2b-46d5-984d-e8200dfd9d9d
> block uuid vqJGf8-V5g0-S1cA-BAcN-Qm9D-9VTx-xsc8wk
> cephx lockbox secret
> cluster fsid id
> cluster name ceph
> crush device class
> db device
>
/dev/ceph-9c39b87c-a39c-413f-b1ef-07881195fcb8/osd-db-feee5095-b5e7-47a0-ae87-8e5039512661
> db uuid Rtj5KA-Qxjk-IFmY-3ffQ-gSAu-Snte-uku8HC
> encrypted 0
> osd fsid 2f009760-fc2b-46d5-984d-e8200dfd9d9d
> osd id 25
> osdspec affinity osd_spec_wall
> type db
> vdo 0
> with tpm 0
> devices /dev/sde
>
>
>
> The other /dev/sde device did show up as data_device instead of
> db_device (example here on host2) :
>
> ceph-volume lvm list
>
> ====== osd.17 ======
>
> [block]
>
/dev/ceph-629f98b0-5ed4-4e75-81b9-e85ca76afb15/osd-block-5d43d683-1f7f-4dc1-935e-6a79745252f9
>
> block device
>
/dev/ceph-629f98b0-5ed4-4e75-81b9-e85ca76afb15/osd-block-5d43d683-1f7f-4dc1-935e-6a79745252f9
> block uuid HQpp1l-x7IB-kA2W-6gWO-BGlM-VN2k-vYf43R
> cephx lockbox secret
> cluster fsid id
> cluster name ceph
> crush device class
> encrypted 0
> osd fsid 5d43d683-1f7f-4dc1-935e-6a79745252f9
> osd id 17
> osdspec affinity osd_spec
> type block
> vdo 0
> with tpm 0
> devices /dev/sde
>
> Thx for your help
> Vivien
>
>
>
>
>
>
>
> _______________________________________________
> ceph-users mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
