Im running two private ceph clusters. Both are cephadm/docker on debian.
The first cluster was upgraded without problem to latest tentacle
(19.2.3 -> 20.2.1) using
ceph orch upgrade start --image quay.io/ceph/ceph:v20.2.1
Today I tried to upgrade the second cluster. During the upgrade, when a
couple of managers had been upgraded, the upgrade process stopped with
the error message
Orchestrator is unavailable: Module not found
and the suggestion I should enable it (ceph mgr module enable
orchestrator). From here Im not exactly certain about the
errors/messages I received, but I could not enable the module (it
already was) and several pages on the dashboard didn't give any
information as it apparently dependent on the orchestrator.
As I neither were near the servers nor had the time to deal with the
failure I chose to shutdown the servers. I have now restarted the
servers and after a while the cluster got up again.
The dashboard (Administration/Upgrade) now shows version 20.2.1 and
"Upgrade is paused". Two managers (incl the active) are upgraded and the
third is old version(19.2.3). Everything else seems old version
(logical, mgr is upgraded first and the error occured during mgr upgrade).
When I got the cluster running again
ceph orch uprade status
showed
{
"in_progress": true,
"target_image":
"quay.io/ceph/ceph@sha256:0bae386bc859cd9a05b804d1ca16cca8853a64f90809044e2bf43095419dc337",
"services_complete": [],
"which": "Upgrading all daemon types on all hosts",
"progress": "2/52 daemons upgraded",
"message": "Error: UPGRADE_EXCEPTION: Upgrade: failed due to an
unexpected exception",
"is_paused": true
}
Any suggestions regarding the continued upgrade? Resume the upgrade from
the dashboard? First manually upgrade the last manager and then resume
the normal upgrade process? or... ?
Anyone seen anything like this before or understand the reason for the
failure?
/Johan
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
