Hi Matthew, thank you for the link to bug #64213! So no fix will be available for V18.2.7..? As I understand the info in the right way there will be a fix in V20 (tentacle)? So should I update to the latest V 19 and wait for V 20.3.0 or stay at 18.2.7 for the moment without any dashboard?
Regards, Christian > Matthew Darwin <b...@mdarwin.ca> hat am 22.08.2025 13:56 CEST geschrieben: > > > Please follow this tracker: https://tracker.ceph.com/issues/64213 > > (basically dashboard does not work on reef/debian) > > On 2025-08-22 07:26, i...@hcpeters.de wrote: > > Hi EUgen > > > > great, that sounds not too bad. > > > > Here is the output: > > ceph health detail > > > > HEALTH_WARN 15 mgr modules have failed dependencies > > [WRN] MGR_MODULE_DEPENDENCY: 15 mgr modules have failed dependencies > > Module 'balancer' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'crash' has failed dependency: PyO3 modules do not yet support > > subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'dashboard' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'devicehealth' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'iostat' has failed dependency: PyO3 modules do not yet support > > subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'nfs' has failed dependency: PyO3 modules do not yet support > > subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'orchestrator' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'pg_autoscaler' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'progress' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'prometheus' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'rbd_support' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'restful' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'status' has failed dependency: PyO3 modules do not yet support > > subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'telemetry' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > Module 'volumes' has failed dependency: PyO3 modules do not yet > > support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576 > > > > Regards, > > > > Christian > > > >> Eugen Block<ebl...@nde.ag> hat am 22.08.2025 12:49 CEST geschrieben: > >> > >> > >> Hi, > >> > >> this is absolutely the right place to ask. > >> > >> First, I would separate the issues (dashboard and crash). And there's > >> no need to redeploy anything, it's just a mgr issue (and crash), > >> nothing critical regarding the operational state of your cluster. > >> > >> Can you paste the output of 'ceph health detail'? And then you should > >> look into the active mgr log and search for stack traces or errors in > >> general. > >> > >> I would ignore crash for now, fix the mgr first. > >> > >> Regards, > >> Eugen > >> > >> Zitat voni...@hcpeters.de: > >> > >>> Hi all, > >>> > >>> not sure it's ok to ask questions here as we are only a little > >>> Theater where I manage the data of our sound and video files. > >>> I set up a ceph cluster as second backup location on bullseye with > >>> quincy which worked flawless. > >>> All runs smooth but I thought I would have to do a step forward to > >>> get on track. > >>> Yes I know: never change a running system. :-D > >>> After updating all three nodes I forget to set the ceph packages on > >>> hold so they were updated to reef with trixie. No way to downgrade > >>> ...I tried a lot....! So I went forward. > >>> After all all 3 nodes are back online running reef my cluster seems > >>> to work, my cephfs share is reachable by my client machine, backups > >>> are working. > >>> This is my ceph status output: > >>> > >>> cluster: > >>> id: c5986871-ab9f-4365-8113-a1854fa07786 > >>> health: HEALTH_WARN > >>> 15 mgr modules have failed dependencies > >>> > >>> services: > >>> mon: 3 daemons, quorum ceph1,ceph2,ceph3 (age 19h) > >>> mgr: ceph1(active, since 18h), standbys: ceph2, ceph3 > >>> mds: 1/1 daemons up, 2 standby > >>> osd: 24 osds: 24 up (since 19h), 24 in (since 22M) > >>> > >>> data: > >>> volumes: 1/1 healthy > >>> pools: 3 pools, 1041 pgs > >>> objects: 3.34M objects, 11 TiB > >>> usage: 32 TiB used, 56 TiB / 87 TiB avail > >>> pgs: 1041 active+clean > >>> > >>> Problem: dashboard is not working any more and I have this erre/warn: > >>> > >>> 15 mgr modules have failed dependencies > >>> > >>> > >>> The cluster was made without podman/containers. > >>> > >>> ceph-crash gives me a flood of this errors: > >>> > >>> ... > >>> WARNING:ceph-crash:post > >>> /var/lib/ceph/crash/2025-08-21T11:51:45.411885Z_738ee242-f7bd-4f6c-bd90-d96a08e06bf6 > >>> as client.admin failed: 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1 > >>> auth: unable to find a keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1 > >>> AuthRegistry(0x7ff728064068) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 > >>> AuthRegistry(0x7ff728068540) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 > >>> AuthRegistry(0x7ff72cea8410) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> [errno 2] RADOS object not found (error connecting to the cluster) > >>> WARNING:ceph-crash:post > >>> /var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa > >>> as client.crash.ceph1 failed: 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 > >>> -1 auth: unable to find a keyring on > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 > >>> AuthRegistry(0x7f81ac0640c8) no keyring found at > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 > >>> AuthRegistry(0x7f81ac0686c0) no keyring found at > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 > >>> AuthRegistry(0x7f81b3ea8410) no keyring found at > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> [errno 2] RADOS object not found (error connecting to the cluster) > >>> WARNING:ceph-crash:post > >>> /var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa > >>> as client.crash failed: 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 > >>> auth: unable to find a keyring on > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 > >>> AuthRegistry(0x7f5b9c064068) no keyring found at > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 > >>> AuthRegistry(0x7f5b9c068540) no keyring found at > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 > >>> AuthRegistry(0x7f5ba2ea8410) no keyring found at > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> [errno 2] RADOS object not found (error connecting to the cluster) > >>> WARNING:ceph-crash:post > >>> /var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa > >>> as client.admin failed: 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1 > >>> auth: unable to find a keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1 > >>> AuthRegistry(0x7fc844064068) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 > >>> AuthRegistry(0x7fc844068540) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 > >>> AuthRegistry(0x7fc8494fd410) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> [errno 2] RADOS object not found (error connecting to the cluster) > >>> WARNING:ceph-crash:post > >>> /var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 > >>> as client.crash.ceph1 failed: 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 > >>> -1 auth: unable to find a keyring on > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 > >>> AuthRegistry(0x7f83d40640c8) no keyring found at > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 > >>> AuthRegistry(0x7f83d40686c0) no keyring found at > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 > >>> AuthRegistry(0x7f83d9aa8410) no keyring found at > >>> /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> [errno 2] RADOS object not found (error connecting to the cluster) > >>> WARNING:ceph-crash:post > >>> /var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 > >>> as client.crash failed: 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1 > >>> auth: unable to find a keyring on > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1 > >>> AuthRegistry(0x7f2a30064068) no keyring found at > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 > >>> AuthRegistry(0x7f2a30068540) no keyring found at > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 > >>> AuthRegistry(0x7f2a37aa8410) no keyring found at > >>> /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> [errno 2] RADOS object not found (error connecting to the cluster) > >>> WARNING:ceph-crash:post > >>> /var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 > >>> as client.admin failed: 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 > >>> auth: unable to find a keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 > >>> AuthRegistry(0x7fa800064068) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 > >>> AuthRegistry(0x7fa800068540) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a > >>> keyring on > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: > >>> (2) No such file or > >>> directory > >>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 > >>> AuthRegistry(0x7fa805ea8410) no keyring found at > >>> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, > >>> disabling > >>> cephx > >>> [errno 2] RADOS object not found (error connecting to the cluster) > >>> ...... > >>> > >>> In /etc/ceph I have > >>> > >>> ceph.client.admin.keyring > >>> ceph.conf > >>> rbdmap > >>> > >>> > >>> Maye there is a way to fix this but if it's to complicated I will > >>> think about a complete re-install. > >>> I'm not an admin but an amateur with some linux knowledge. > >>> If there is hope and someone could support to solev this I would be > >>> very happy! > >>> > >>> Regards, > >>> > >>> Christian Peters > >>> > >>> > >>> > >>> _______________________________________________ > >>> ceph-users mailing list --ceph-users@ceph.io > >>> To unsubscribe send an email toceph-users-le...@ceph.io > >> > >> _______________________________________________ > >> ceph-users mailing list --ceph-users@ceph.io > >> To unsubscribe send an email toceph-users-le...@ceph.io > > _______________________________________________ > > ceph-users mailing list --ceph-users@ceph.io > > To unsubscribe send an email toceph-users-le...@ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
