Public bug reported:
Following the upstream docs (1) to deploy a cephadm based Ceph cluster
on Resolute, results into OSD devices in ERROR state
The ceph status output is similar to the following:
$ sudo ceph -s
cluster:
id: e4638e32-4eaf-11f1-8000-fa163ebbf652
health: HEALTH_WARN
20 failed cephadm daemon(s)
Reduced data availability: 1 pg inactive
services:
mon: 1 daemons, quorum tentacle-mon-1 (age 3h) [leader: tentacle-mon-1]
mgr: tentacle-mon-1.skfhxy(active, since 3h)
osd: 3 osds: 0 up, 1 in (since 45m)
data:
pools: 1 pools, 1 pgs
objects: 0 objects, 0 B
usage: 0 B used, 0 B / 0 B avail
pgs: 100.000% pgs unknown
1 unknown
Looking into the device list:
$ sudo ceph orch device ls
HOST PATH TYPE DEVICE ID
SIZE AVAILABLE REFRESHED REJECT REASONS
tentacle-osd-1 /dev/sdb hdd
QEMU_QEMU_HARDDISK_6cff029e-4ba2-4cee-8dc2-9a8fcbea11c8 500G No 25m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
tentacle-osd-2 /dev/sdb hdd
QEMU_QEMU_HARDDISK_59bf431e-0770-4a19-b375-fb828f2f768c 500G No 22m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
tentacle-osd-3 /dev/sdb hdd
QEMU_QEMU_HARDDISK_8fcbcc88-6fce-4eb0-9520-b2500737fda0 500G No 23m
ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
Digging deeper into the OSD nodes, the podman pods are in a continuous
crash loop:
May 13 13:35:41 tentacle-osd-3 podman[35361]: 2026-05-13 13:35:41.509313996
+0000 UTC m=+0.093476165 container create
b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569
(image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d
1ca16cca8853a64f90809044e2bf43095419dc337, name=dreamy_pascal,
org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True,
CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, CEPH_REF=tentacle,
org.opencontainers.image
.authors=Ceph Release Team <[email protected]>,
org.opencontainers.image.documentation=https://docs.ceph.com/,
OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image,
io.buildah.version=1.43.0, FROM_IMAGE=quay.io/cent
os/centos:stream9, org.label-schema.license=GPLv2,
org.label-schema.build-date=20260331,
GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/,
CEPH_GIT_REPO=https://github.com/ceph/ce
ph.git)
May 13 13:35:41 tentacle-osd-3 podman[35361]: 2026-05-13 13:35:41.446666907
+0000 UTC m=+0.030829335 image pull
fb63cba66eea5516fbd9a1ec035c265abdbc1a1a8fc2dc0b51057557e49ca28c
quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a
64f90809044e2bf43095419dc337
May 13 13:35:41 tentacle-osd-3 systemd[1]: Started
libpod-conmon-b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569.scope.
May 13 13:35:41 tentacle-osd-3 systemd[1]: Started
libpod-b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569.scope -
libcrun container.
May 13 13:35:41 tentacle-osd-3 podman[35361]: 2026-05-13 13:35:41.621518902
+0000 UTC m=+0.205681252 container init
b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569
(image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1c
a16cca8853a64f90809044e2bf43095419dc337, name=dreamy_pascal, CEPH_REF=tentacle,
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS,
org.label-schema.schema-version=1.0, io.buildah.version=1.43.0,
GANESHA_REPO_BASEURL=https://b
uildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/,
OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image,
FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True,
org.opencontainers.image.documentati
on=https://docs.ceph.com/, org.label-schema.build-date=20260331,
CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2,
CEPH_GIT_REPO=https://github.com/ceph/ceph.git,
org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers@ceph
.io>)
May 13 13:35:41 tentacle-osd-3 podman[35361]: 2026-05-13 13:35:41.629197416
+0000 UTC m=+0.213359966 container start
b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569
(image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1
ca16cca8853a64f90809044e2bf43095419dc337, name=dreamy_pascal,
io.buildah.version=1.43.0, org.label-schema.license=GPLv2,
org.label-schema.build-date=20260331,
GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/sto
rage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0,
org.opencontainers.image.authors=Ceph Release Team <[email protected]>,
org.label-schema.vendor=CentOS, CEPH_REF=tentacle, ceph=True,
CEPH_SHA1=6a49aff47758778a5f59
51e731d437c317f72fb2, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9
Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/,
CEPH_GIT_REPO=https://github.com/ceph/ceph.git,
FROM_IMAGE=quay.io/centos/centos:st
ream9)
May 13 13:35:41 tentacle-osd-3 podman[35361]: 2026-05-13 13:35:41.6674013 +0000
UTC m=+0.251563551 container attach
b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569
(image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1c
a16cca8853a64f90809044e2bf43095419dc337, name=dreamy_pascal,
OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image,
FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0,
CEPH_GIT_REPO=https://github
.com/ceph/ceph.git, ceph=True,
org.opencontainers.image.documentation=https://docs.ceph.com/,
org.label-schema.license=GPLv2, org.label-schema.build-date=20260331,
CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, org.opencontainers.ima
ge.authors=Ceph Release Team <[email protected]>,
io.buildah.version=1.43.0, org.label-schema.vendor=CentOS, CEPH_REF=tentacle,
GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesh
a-5/)
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: [
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: {
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "available": false,
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "being_replaced":
false,
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "ceph_device_lvm":
true,
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "device_id":
"QEMU_QEMU_HARDDISK_8fcbcc88-6fce-4eb0-9520-b2500737fda0",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "lsm_data": {},
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "lvs": [
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: {
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"block_uuid": "gBOeRn-eROy-O3C3-3m1a-YHlW-jGN9-raiVts",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"cluster_fsid": "e4638e32-4eaf-11f1-8000-fa163ebbf652",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"cluster_name": "ceph",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "name":
"osd-block-00a964ee-4d94-40a8-913a-b8f22e8e51c6",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"osd_fsid": "00a964ee-4d94-40a8-913a-b8f22e8e51c6",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "osd_id":
"1",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"osdspec_affinity": "all-available-devices",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "type":
"block"
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: }
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: ],
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "path": "/dev/sdb",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"rejected_reasons": [
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "Insufficient
space (<10 extents) on vgs",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "LVM detected",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "Has a
FileSystem"
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: ],
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "sys_api": {
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "actuators":
null,
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"device_nodes": [
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "sdb"
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: ],
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "devname":
"sdb",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"human_readable_size": "500.00 GB",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "id_bus":
"scsi",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "model": "QEMU
HARDDISK",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "nr_requests":
"256",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "parent":
"/dev/sdb",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "partitions":
{},
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "path":
"/dev/sdb",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "removable":
"0",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "rev": "2.5+",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "ro": "0",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "rotational":
"1",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "sas_address":
"",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"sas_device_handle": "",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"scheduler_mode": "none",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "sectors": 0,
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "sectors": 0,
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "sectorsize":
"512",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "size":
536870912000.0,
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]:
"support_discard": "4096",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "type": "disk",
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: "vendor":
"QEMU"
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: }
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: }
May 13 13:35:42 tentacle-osd-3 dreamy_pascal[35371]: ]
May 13 13:35:42 tentacle-osd-3 systemd[1]:
libpod-b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569.scope:
Deactivated successfully.
May 13 13:35:42 tentacle-osd-3 conmon[35371]: conmon b563b6dde70ca5890af4
<nwarn>: Failed to open cgroups file:
/sys/fs/cgroup/machine.slice/libpod-b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569.scope/container/memory.ev
ents
May 13 13:35:42 tentacle-osd-3 podman[35361]: 2026-05-13 13:35:42.503989777
+0000 UTC m=+1.088152030 container died
b563b6dde70ca5890af455e8ddd1d4c069a7a53a152c625fc4e74a77e64b4569
(image=quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1c
a16cca8853a64f90809044e2bf43095419dc337, name=dreamy_pascal,
OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9,
org.label-schema.vendor=CentOS, CEPH_REF=tentacle,
CEPH_SHA1=6a49aff47758778a5f5951e731d437c317f72fb2, CEPH_GIT_REP
O=https://github.com/ceph/ceph.git, org.label-schema.schema-version=1.0,
org.opencontainers.image.authors=Ceph Release Team <[email protected]>,
io.buildah.version=1.43.0,
org.opencontainers.image.documentation=https://docs.ceph.com/,
org.label-schema.build-date=20260331, org.label-schema.name=CentOS Stream 9
Base Image, ceph=True, org.label-schema.license=GPLv2,
GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesh
a-5/)
May 13 13:35:42 tentacle-osd-3 systemd[1]:
var-lib-containers-storage-overlay-f823b1c45f9c1449a945538ef8edd666b5ecc5981593e9d706c2caeed0e0276f-merged.mount:
Deactivated successfully.
More investigation is needed, but this may be related to cgroup changes in
Resolute.
Thanks,
Alan
(1) https://docs.ceph.com/en/latest/cephadm/install/
** Affects: ceph (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2152555
Title:
Ceph 20.2.0 - cephadm osd bootstrap fails
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2152555/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
