Hi, Just guessing - it's Lets Encrypt certs? Or may be something InternalCA when certificate issues by golang application?
k Sent from my iPhone > On 24 Apr 2026, at 15:54, Flemming Frandsen via ceph-users > <[email protected]> wrote: > > I have a tentacle 20.2.1 cluster managed by cephadm, where I have > configured a certificate for the rgw using something like: > > service_type: ingress > service_id: rgw.default > service_name: ingress.rgw.default > placement: > count: 3 > label: rgw > spec: > backend_service: rgw.default > first_virtual_router_id: 50 > frontend_port: 443 > monitor_port: 8080 > virtual_ip: 10.0.40.3/24 > networks: > - 10.0.40.0/24 > ssl_cert: | > -----BEGIN CERTIFICATE----- > *snip* > -----END CERTIFICATE----- > -----BEGIN CERTIFICATE----- > *snip* > -----END CERTIFICATE----- > -----BEGIN RSA PRIVATE KEY----- > *snip* > -----END RSA PRIVATE KEY----- > > This initially worked perfectly, but I have just rotated the certificate by > generating the yaml with a fresh private key and associated certificate > then installing it with: > ceph orch apply -i new.yaml > > Clients accessing the rgw ingress get the correct certificate and work > perfectly, both before and after the key rotation. > > > I have a script that does this for me daily, and until the certificate was > rotated, everything was fine, but then I got this error: > > # ceph orch certmgr cert check > > Certificate 'ingress_ssl_cert (ingress.rgw.default)' (user-made) is not > valid (error: Invalid cert/key pair: [('x509 certificate routines', '', > 'key values mismatch')]) > > After messing with the certificate a bunch I now get a completely > different, but also worrying error: > > # ceph orch certmgr cert check > - Certificate 'ingress_ssl_cert (ingress.rgw.default)' (user-made) is not > valid (error: Invalid private key: [('DECODER routines', '', > 'unsupported')]) > > The documentation offers no help: > https://docs.ceph.com/en/tentacle/cephadm/certmgr/ > > The error seems to come from this code: > https://github.com/ceph/ceph/blob/main/src/python-common/ceph/cryptotools/internal.py#L134 > > So the error is an amputated exception from somewhere in openssl but the > actual exception is eaten by the code, so there is no way for me to debug > the situation: > https://github.com/ceph/ceph/blob/main/src/python-common/ceph/cryptotools/internal.py#L134 > > I have tried installing the old certificate and key, that only leads the > the Invalid private key: [('DECODER routines', '', 'unsupported')] error. > > I have tried re-formatting the pkcs#1 RSA key as PKCS#8, but that does > nothing. > > I have even tried copying the key and cert pem files into the mgr container > and verified them using the openssl installed in the container. > > The certificates are generated by cert-manager, running in k8s and a > cronjob that periodically pushes the new key/cert to the ceph cluster via > ssh. > > Has anyone seen these types of errors before? > > -- > Flemming Frandsen - YAPH - http://osaa.dk - http://dren.dk/ > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
