On Fri, Sep 22, 2017 at 10:35 AM, Martin Perina <[email protected]> wrote: > > > On Fri, Sep 22, 2017 at 10:18 AM, Neil <[email protected]> wrote: >> >> Hi Piotr, >> >> Thank you for the information. >> >> It looks like something has expired looking in the server.log now that >> debug is enabled. >> >> 2017-09-22 09:35:26,462 INFO [stdout] (MSC service thread 1-4) Version: >> V3 >> 2017-09-22 09:35:26,464 INFO [stdout] (MSC service thread 1-4) Subject: >> CN=engine01.mydomain.za, O=mydomain, C=US >> 2017-09-22 09:35:26,467 INFO [stdout] (MSC service thread 1-4) >> Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5 >> 2017-09-22 09:35:26,471 INFO [stdout] (MSC service thread 1-4) >> 2017-09-22 09:35:26,472 INFO [stdout] (MSC service thread 1-4) Key: >> Sun RSA public key, 1024 bits >> 2017-09-22 09:35:26,474 INFO [stdout] (MSC service thread 1-4) modulus: >> 96670613185023785772001656613227416922514371649313203413281121371175732119596513752882171306045450346018887835032223373125981220753972276294203593174404470265593368091683564110524316403260121331609213962612618181708680331850541390318868926054438078223371655800890725486783860059873397983318033852172060923531 >> 2017-09-22 09:35:26,476 INFO [stdout] (MSC service thread 1-4) public >> exponent: 65537 >> 2017-09-22 09:35:26,477 INFO [stdout] (MSC service thread 1-4) >> Validity: [From: Sun Oct 14 22:26:46 SAST 2012, >> 2017-09-22 09:35:26,478 INFO [stdout] (MSC service thread 1-4) >> To: Tue Sep 19 18:26:49 SAST 2017] >> 2017-09-22 09:35:26,479 INFO [stdout] (MSC service thread 1-4) Issuer: >> CN=CA-engine01.mydomain.za.47472, O=mydomain, C=US >> >> Any idea how I can generate a new one and what cert it is that's expired? > > > It seems that your engine certificate has expired, but AFAIK this > certificate should be automatically renewed during engine-setup. So when did > you execute engine-setup for last time? Any info/warning about this shown > during invocation?
Correct, Martin was a bit faster then me :) > > Also looking at server.log I found JBoss 7.1.1, so you are using really > ancient oVirt, version, right? > >> >> Please see the attached log for more info. >> >> Thank you so much for your assistance. >> >> Regards. >> >> Neil Wilson. >> >> >> >> >> >> >> On Thu, Sep 21, 2017 at 8:41 PM, Piotr Kliczewski >> <[email protected]> wrote: >>> >>> Neil, >>> >>> It seems that your engine certificate(s) is/are not ok. I would >>> suggest to enable ssl debug in the engine by: >>> - add '-Djavax.net.debug=all' to ovirt-engine.py file here [1]. >>> - restart your engine >>> - check your server.log and check what is the issue. >>> >>> Hopefully we will be able to understand what happened in your setup. >>> >>> Thanks, >>> Piotr >>> >>> [1] >>> https://github.com/oVirt/ovirt-engine/blob/master/packaging/services/ovirt-engine/ovirt-engine.py#L341 >>> >>> On Thu, Sep 21, 2017 at 4:42 PM, Neil <[email protected]> wrote: >>> > Further to the logs sent, on the nodes I'm also seeing the following >>> > error >>> > under /var/log/messages... >>> > >>> > Sep 20 03:43:12 node01 vdsm root ERROR invalid client certificate with >>> > subject "/C=US/O=UKDM/CN=engine01.mydomain.za"^C >>> > Sep 20 03:43:12 node01 vdsm vds ERROR xml-rpc handler >>> > exception#012Traceback >>> > (most recent call last):#012 File "/usr/share/vdsm/BindingXMLRPC.py", >>> > line >>> > 80, in threaded_start#012 self.server.handle_request()#012 File >>> > "/usr/lib64/python2.6/SocketServer.py", line 278, in handle_request#012 >>> > self._handle_request_noblock()#012 File >>> > "/usr/lib64/python2.6/SocketServer.py", line 288, in >>> > _handle_request_noblock#012 request, client_address = >>> > self.get_request()#012 File "/usr/lib64/python2.6/SocketServer.py", >>> > line >>> > 456, in get_request#012 return self.socket.accept()#012 File >>> > "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line >>> > 136, >>> > in accept#012 raise SSL.SSLError("%s, client %s" % (e, >>> > address[0]))#012SSLError: no certificate returned, client 10.251.193.5 >>> > >>> > Not sure if this is any further help in diagnosing the issue? >>> > >>> > Thanks, any assistance is appreciated. >>> > >>> > Regards. >>> > >>> > Neil Wilson. >>> > >>> > >>> > On Thu, Sep 21, 2017 at 4:31 PM, Neil <[email protected]> wrote: >>> >> >>> >> Hi Piotr, >>> >> >>> >> Thank you for the reply. After sending the email I did go and check >>> >> the >>> >> engine one too.... >>> >> >>> >> [root@engine01 /]# openssl x509 -in /etc/pki/ovirt-engine/ca.pem >>> >> -enddate >>> >> -noout >>> >> notAfter=Oct 13 16:26:46 2022 GMT >>> >> >>> >> I'm not sure if this one below is meant to verify or if this output is >>> >> expected? >>> >> >>> >> [root@engine01 /]# openssl x509 -in >>> >> /etc/pki/ovirt-engine/private/ca.pem >>> >> -enddate -noout >>> >> unable to load certificate >>> >> 140642165552968:error:0906D06C:PEM routines:PEM_read_bio:no start >>> >> line:pem_lib.c:703:Expecting: TRUSTED CERTIFICATE >>> >> >>> >> My date is correct too Thu Sep 21 16:30:15 SAST 2017 >>> >> >>> >> Any ideas? >>> >> >>> >> Googling surprisingly doesn't come up with much. >>> >> >>> >> Thank you. >>> >> >>> >> Regards. >>> >> >>> >> Neil Wilson. >>> >> >>> >> On Thu, Sep 21, 2017 at 4:16 PM, Piotr Kliczewski >>> >> <[email protected]> wrote: >>> >>> >>> >>> Neil, >>> >>> >>> >>> You checked both nodes what about the engine? Can you check engine >>> >>> certs? >>> >>> You can find more info where they are located here [1]. >>> >>> >>> >>> Thanks, >>> >>> Piotr >>> >>> >>> >>> [1] >>> >>> >>> >>> https://www.ovirt.org/develop/release-management/features/infra/pki/#ovirt-engine >>> >>> >>> >>> On Thu, Sep 21, 2017 at 3:26 PM, Neil <[email protected]> wrote: >>> >>> > Hi guys, >>> >>> > >>> >>> > Please could someone assist, my cluster is down and I can't access >>> >>> > my >>> >>> > vm's >>> >>> > to switch some of them back on. >>> >>> > >>> >>> > I'm seeing the following error in the engine.log however I've >>> >>> > checked >>> >>> > my >>> >>> > certs on my hosts (as some of the goolge results said to check), >>> >>> > but >>> >>> > the >>> >>> > certs haven't expired... >>> >>> > >>> >>> > >>> >>> > 2017-09-21 15:09:45,077 ERROR >>> >>> > >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] >>> >>> > (DefaultQuartzScheduler_Worker-4) Command >>> >>> > GetCapabilitiesVDSCommand(HostName >>> >>> > = node02.mydomain.za, HostId = >>> >>> > d2debdfe-76e7-40cf-a7fd-78a0f50f14d4, >>> >>> > vds=Host[node02.mydomain.za]) execution failed. Exception: >>> >>> > VDSNetworkException: javax.net.ssl.SSLHandshakeException: Received >>> >>> > fatal >>> >>> > alert: certificate_expired >>> >>> > 2017-09-21 15:09:45,086 ERROR >>> >>> > >>> >>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] >>> >>> > (DefaultQuartzScheduler_Worker-10) Command >>> >>> > GetCapabilitiesVDSCommand(HostName = node01.mydomain.za, HostId = >>> >>> > b108549c-1700-11e2-b936-9f5243b8ce13, vds=Host[node01.mydomain.za]) >>> >>> > execution failed. Exception: VDSNetworkException: >>> >>> > javax.net.ssl.SSLHandshakeException: Received fatal alert: >>> >>> > certificate_expired >>> >>> > 2017-09-21 15:09:48,173 ERROR >>> >>> > >>> >>> > My engine and host info is below... >>> >>> > >>> >>> > [root@engine01 ovirt-engine]# rpm -qa | grep -i ovirt >>> >>> > ovirt-engine-lib-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-restapi-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-setup-plugin-websocket-proxy-3.4.0-1.el6.noarch >>> >>> > ovirt-host-deploy-java-1.2.0-1.el6.noarch >>> >>> > ovirt-engine-setup-3.4.0-1.el6.noarch >>> >>> > ovirt-host-deploy-1.2.0-1.el6.noarch >>> >>> > ovirt-engine-backend-3.4.0-1.el6.noarch >>> >>> > ovirt-image-uploader-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-tools-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-sdk-python-3.4.0.7-1.el6.noarch >>> >>> > ovirt-engine-webadmin-portal-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-cli-3.4.0.5-1.el6.noarch >>> >>> > ovirt-engine-setup-base-3.4.0-1.el6.noarch >>> >>> > ovirt-iso-uploader-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-userportal-3.4.0-1.el6.noarch >>> >>> > ovirt-log-collector-3.4.1-1.el6.noarch >>> >>> > ovirt-engine-websocket-proxy-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-setup-plugin-ovirt-engine-common-3.4.0-1.el6.noarch >>> >>> > ovirt-engine-dbscripts-3.4.0-1.el6.noarch >>> >>> > [root@engine01 ovirt-engine]# cat /etc/redhat-release >>> >>> > CentOS release 6.5 (Final) >>> >>> > >>> >>> > >>> >>> > [root@node02 ~]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem >>> >>> > -enddate >>> >>> > -noout ; date >>> >>> > notAfter=May 27 08:36:17 2019 GMT >>> >>> > Thu Sep 21 15:18:22 SAST 2017 >>> >>> > CentOS release 6.5 (Final) >>> >>> > [root@node02 ~]# rpm -qa | grep vdsm >>> >>> > vdsm-4.14.6-0.el6.x86_64 >>> >>> > vdsm-python-4.14.6-0.el6.x86_64 >>> >>> > vdsm-cli-4.14.6-0.el6.noarch >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch >>> >>> > >>> >>> > >>> >>> > [root@node01 ~]# openssl x509 -in /etc/pki/vdsm/certs/vdsmcert.pem >>> >>> > -enddate >>> >>> > -noout ; date >>> >>> > notAfter=Jun 13 16:09:41 2018 GMT >>> >>> > Thu Sep 21 15:18:52 SAST 2017 >>> >>> > CentOS release 6.5 (Final) >>> >>> > [root@node01 ~]# rpm -qa | grep -i vdsm >>> >>> > vdsm-4.14.6-0.el6.x86_64 >>> >>> > vdsm-xmlrpc-4.14.6-0.el6.noarch >>> >>> > vdsm-cli-4.14.6-0.el6.noarch >>> >>> > vdsm-python-zombiereaper-4.14.6-0.el6.noarch >>> >>> > vdsm-python-4.14.6-0.el6.x86_64 >>> >>> > >>> >>> > Please could I have some assistance, I'm rater desperate. >>> >>> > >>> >>> > Thank you. >>> >>> > >>> >>> > Regards. >>> >>> > >>> >>> > Neil Wilson >>> >>> > >>> >>> > >>> >>> > >>> >>> > _______________________________________________ >>> >>> > Users mailing list >>> >>> > [email protected] >>> >>> > http://lists.ovirt.org/mailman/listinfo/users >>> >>> > >>> >> >>> >> >>> > >> >> >> >> _______________________________________________ >> Users mailing list >> [email protected] >> http://lists.ovirt.org/mailman/listinfo/users >> > _______________________________________________ Users mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/users
