Hi,
this sounds familiar:
https://lists.ceph.io/hyperkitty/list/[email protected]/thread/Y2YKHQZSBYA7GS4D4YSPSNMWNE5QMETH/#ZTUQ2CE5547LOPBGQOXHUF63F6UU3RKO
You could set the OSD "in" and then retry starting it a couple of times.
Regards,
Eugen
Zitat von Gilles Mocellin via ceph-users <[email protected]>:
Hello,
After removing a failing drive with :
ceph orch osd rm ${ID} --zap --replace
Waiting for recovery (3 days, in a 12 nodes 144 drives)...
I add the drive with :
ceph orch daemon add osd myserver:/dev/sdn
In the journal logs of the OSD, I can see :
Jun 23 10:06:51 myserver bash[1944632]: debug
2026-06-23T08:06:51.908+0000 7f1fffa90740 1 osd.66 389707 start_boot
Jun 23 10:06:51 myserver bash[1944632]: debug
2026-06-23T08:06:51.908+0000 7f1fffa90740 1 osd.66 389707
maybe_override_options_for_qos osd_max_backfills set to 1
Jun 23 10:06:51 myserver bash[1944632]: debug
2026-06-23T08:06:51.912+0000 7f1fffa90740 1 osd.66 389707
maybe_override_options_for_qos osd_recovery_max_active set to 0
Jun 23 10:06:51 myserver bash[1944632]: debug
2026-06-23T08:06:51.912+0000 7f1fffa90740 1 osd.66 389707
maybe_override_options_for_qos osd_recovery_max_active_hdd set to 3
Jun 23 10:06:51 myserver bash[1944632]: debug
2026-06-23T08:06:51.912+0000 7f1fffa90740 1 osd.66 389707
maybe_override_options_for_qos osd_recovery_max_active_ssd set to 10
Jun 23 10:06:51 myserver bash[1944632]: debug
2026-06-23T08:06:51.912+0000 7f1fffa90740 1 osd.66 389707 bench
count 12288000 bsize 4 KiB
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.552+0000 7f1fffa90740 1 osd.66 389707
maybe_override_max_osd_capacity_for_qos osd bench result - bandwidth
(MiB/sec): 6.518 iops: 1668.622 elapsed_sec: 1.798
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.552+0000 7f1fffa90740 0 log_channel(cluster)
log [WRN] : OSD bench result of 1668.621657 IOPS is not within the
threshold limit range of 50.000000 IOPS and 500.000000 IOPS for
osd.66. IOPS capacity is unchanged at 315.000000 IOPS. The
recommendation is to establish the osd's IOPS capacity using other
benchmark tools (e.g. Fio) and then override
osd_mclock_max_capacity_iops_[hdd|ssd].
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.552+0000 7f1ffba10640 -1 osd.66 389707 osdmap
says I am destroyed
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.740+0000 7f1fef00a640 -1 osd.66 389747 map says
i am stopped by admin. shutting down.
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.740+0000 7f1fef00a640 0 osd.66 389747
_committed_osd_maps shutdown OSD via async signal
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.740+0000 7f1ffca28640 -1 received signal:
Interrupt from Kernel ( Could be generated by pthread_kill(),
raise(), abort(), alarm() ) UID: 0
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.740+0000 7f1ffca28640 -1 osd.66 389747 *** Got
signal Interrupt ***
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.740+0000 7f1ffca28640 0 osd.66 389747 Fast
Shutdown: - cct->_conf->osd_fast_shutdown = 1, null-fm = 0
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.740+0000 7f1ffca28640 -1 osd.66 389747 ***
Immediate shutdown (osd_fast_shutdown=true) ***
Jun 23 10:06:55 myserver bash[1944632]: debug
2026-06-23T08:06:55.740+0000 7f1ffca28640 0 osd.66 389747
prepare_to_stop starting shutdown
Jun 23 10:06:57 myserver systemd[1]:
[email protected]:
Deactivated successfully.
I retry several time, nothing new.
The OSD is mark down and then stops.
I've found that solution on RedHat site :
https://access.redhat.com/solutions/7029908
Which is related to epochs, they have that log before :
276003 handle_osd_map epochs [276004,276043] i have 276003, src has
[274883, 303268]
276043 map says i am stopped by admin. shutting down.
I don't have messages about epochs.
I tried anyway with thoses settings :
# ceph config set osd osd_map_message_max 1000
# ceph config set osd osd_map_share_max_epochs 1000
But no, the OSD still doesn't start.
Has anyone a clue ?
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
