Could dig bit deeper into the issue and found one resource online where a fix is suggested to add following into ambari-agent.ini
force_https_protocol=PROTOCOL_TLSv1_2 (source: http://knowledge.teradata.com/KCS/id/KCS008843) On doing so, I am getting following error: INFO 2017-11-20 10:18:39,540 NetUtil.py:70 - Connecting to https://ambari-server-host:8440/ca WARNING 2017-11-20 10:18:39,541 NetUtil.py:101 - Failed to connect to https://ambari-server-host:8440/ca due to 'module' object has no attribute 'PROTOCOL_TLSv1_2' WARNING 2017-11-20 10:18:39,541 NetUtil.py:124 - Server at https://ambari-server-host:8440 is not reachable, sleeping for 10 seconds... INFO 2017-11-20 10:18:49,541 NetUtil.py:70 - Connecting to https://ambari-server-host:8440/ca *I am using python 2.6.6 and it seems ssl.PROTOCOL_TLSv1_2 was added in python 2.7.9* Using openssl s_client, I am successfully able to connect to the server and get certificate. *openssl s_client -connect ambari-server-host:8441 -tls1_2* 7iAh9/YddjuVxLzd1wLhpoEDcGPNj6HFkSu/zGucYV2F3eshEdIIqDFiE177styv 3og0SjEvNuIwa1ECYr+9Qm5yYk82NAowf5cQcx+ykboEsFMfBkvmmw0JhZMp1OB5 qnYL1cGI4Sp55w== -----END CERTIFICATE----- ... --- ... --- SSL handshake has read 3184 bytes and written 338 bytes --- New, TLSv1/SSLv3, Cipher is DES-CBC3-SHA Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.1 Cipher : DES-CBC3-SHA Session-ID: 5A12BA5D9A2296C2B43BD1869D2DEFD97B334D0A07C843613206D66AACA7E99A Session-ID-ctx: Master-Key: AA96C013DE644728F03D994B318A5785760DFADAB622323A21E8D593F41485943495B810C78CA246A05A892012BEA11D Key-Arg : None Krb5 Principal: None PSK identity: None PSK identity hint: None Start Time: 1511176797 Timeout : 7200 (sec) Verify return code: 0 (ok) But agent (python code) fails to do so [logs snippet below] INFO 2017-11-20 10:39:29,799 security.py:93 - SSL Connect being called.. connecting to the server ERROR 2017-11-20 10:39:29,815 security.py:80 - Two-way SSL authentication failed. Ensure that server and agent certificates were signed by the same CA and restart the agent. In order to receive a new agent certificate, remove existing certificate file from keys directory. As a workaround you can turn off two-way SSL authentication in server configuration(ambari.properties) Exiting.. ERROR 2017-11-20 10:39:29,815 Controller.py:226 - Unable to connect to: https://ambari-server-host:8441/agent/v1/register/ambari-agent-host Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/ambari_agent/Controller.py", line 175, in registerWithServer ret = self.sendRequest(self.registerUrl, data) File "/usr/lib/python2.6/site-packages/ambari_agent/Controller.py", line 549, in sendRequest raise IOError('Request to {0} failed due to {1}'.format(url, str(exception))) *IOError: Request to https://ambari-server-host:8441/agent/v1/register/ambari-agent-host <https://ambari-server-host:8441/agent/v1/register/ambari-agent-host> failed due to [Errno 1] _ssl.c:492: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed* *ERROR 2017-11-20 10:39:29,816 Controller.py:227 - Error:Request to https://ambari-server-host:8441/agent/v1/register/ambari-agent-host <https://ambari-server-host:8441/agent/v1/register/ambari-agent-host> failed due to [Errno 1] _ssl.c:492: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed* WARNING 2017-11-20 10:39:29,816 Controller.py:228 - Sleeping for 7 seconds and then trying again Currently, *It's not possible for us to have python 2.7.9 installed on our production machines*. Is there any other way/alternative to move past this problem? and Use Two-Way SSL in production. Regards, Sandeep. On Tue, Nov 14, 2017 at 8:12 AM, Sandy <[email protected]> wrote: > Yeah. I had put Root CA cert CA.crt in server keys directory and added > agent cert to to keystore too. > > Followed the exact instructions given in link above. > > I'm using just two machine setup. One server and one agent machine. > > Sandeep. > > > On Mon, 13 Nov 2017 at 9:08 PM, Gonzalo Herreros <[email protected]> > wrote: > >> Did you also put the ca.crt file on the server >> /var/lib/ambari-server/keys? >> Also in the steps you don't mention adding all the certs to the server >> keystore which is in the instructions >> >> I think it's much less error prone to let ambari create and assign the >> certs it uses to communicate with agents (being careful to configure >> longer >> life) >> >> >> On 13 November 2017 at 14:54, Sandy <[email protected]> wrote: >> >> > We are trying to setup security in server agent communication using >> > instructions provided in this >> > >> > https://community.hortonworks.com/articles/107092/configure- >> > 2-way-ssl-between-ambari-server-and-amba.html >> > >> > Here are the things I've done: >> > 1. Obtained Certificates from CA for both server and agent machines. >> > 2. Placed them in corresponding directories in both server and agent. >> > 3. When we try to manually verify authentication using following >> command, >> > two way ssl seems to be working (*openssl s_client -cert >> > agent-hostname.crt -key **agent-hostname**.key >> > -CAfile /var/lib/ambari-agent/keys/ca.crt -connect server-hostname:8441 >> > -msg) Detailed logs below* >> > 4. But ambari agent throws following error >> > >> > INFO 2017-11-13 14:36:43,411 NetUtil.py:70 - Connecting to >> > https://server-hostname:8440/connection_info >> > >> > INFO 2017-11-13 14:36:43,426 security.py:55 - Server require two-way SSL >> > authentication. Use it instead of one-way... >> > >> > INFO 2017-11-13 14:36:43,426 security.py:183 - Server certicate exists, >> ok >> > >> > INFO 2017-11-13 14:36:43,426 security.py:191 - Agent key exists, ok >> > >> > INFO 2017-11-13 14:36:43,427 security.py:199 - Agent certificate >> exists, ok >> > >> > INFO 2017-11-13 14:36:43,427 security.py:94 - SSL Connect being called.. >> > connecting to the server >> > >> > ERROR 2017-11-13 14:36:43,432 security.py:81 - Two-way SSL >> authentication >> > failed. Ensure that server and agent certificates were signed by the >> same >> > CA and restart the agent. >> > >> > In order to receive a new agent certificate, remove existing certificate >> > file from keys directory. As a workaround you can turn off two-way SSL >> > authentication in server configuration(ambari.properties) >> > >> > Exiting.. >> > >> > ERROR 2017-11-13 14:36:43,432 Controller.py:226 - Unable to connect to: >> > https://server-hostname:8441/agent/v1/register/agent-hostname >> > >> > Traceback (most recent call last): >> > >> > File "/usr/lib/python2.6/site-packages/ambari_agent/Controller.py", >> line >> > 175, in registerWithServer >> > >> > ret = self.sendRequest(self.registerUrl, data) >> > >> > File "/usr/lib/python2.6/site-packages/ambari_agent/Controller.py", >> line >> > 549, in sendRequest >> > >> > raise IOError('Request to {0} failed due to {1}'.format(url, str >> > (exception))) >> > >> > IOError: Request to >> > https://server-hostname:8441/agent/v1/register/agent-hostname failed >> due >> > to >> > [Errno 8] _ssl.c:492: EOF occurred in violation of protocol >> > >> > ERROR 2017-11-13 14:36:43,433 Controller.py:227 - Error:Request to >> > https://server-hostname:8441/agent/v1/register/agent-hostname failed >> due >> > to >> > [Errno 8] _ssl.c:492: EOF occurred in violation of protocol >> > >> > WARNING 2017-11-13 14:36:43,433 Controller.py:228 - Sleeping for 11 >> > seconds and then trying again >> > >> > >> > *Can someone help ?* >> > >> > -bash-4.1$ *openssl s_client -cert agent-hostname.crt -key >> > **agent-hostname**.key >> > -CAfile /var/lib/ambari-agent/keys/ca.crt -connect server-hostname:8441 >> > -msg * >> > >> > CONNECTED(00000003) >> > >> > >>> TLS 1.2 Handshake [length 00f2], ClientHello >> > >> > ... >> > >> > <<< TLS 1.2 Handshake [length 0051], ServerHello >> > >> > ... >> > >> > <<< TLS 1.2 Handshake [length 0524], Certificate >> > >> > ... >> > >> > verify return:1 >> > >> > <<< TLS 1.2 Handshake [length 0191], ServerKeyExchange >> > >> > ... >> > >> > <<< TLS 1.2 Handshake [length 00d2], CertificateRequest >> > >> > ... >> > >> > <<< TLS 1.2 Handshake [length 0004], ServerHelloDone >> > >> > 0e 00 00 00 >> > >> > >>> TLS 1.2 Handshake [length 0fe2], Certificate >> > >> > ... >> > >> > >>> TLS 1.2 Handshake [length 008a], ClientKeyExchange >> > >> > ... >> > >> > >>> TLS 1.2 Handshake [length 0108], CertificateVerify >> > >> > ... >> > >> > >>> TLS 1.2 ChangeCipherSpec [length 0001] >> > >> > 01 >> > >> > >>> TLS 1.2 Handshake [length 0010], Finished >> > >> > --- >> > >> > Certificate chain >> > >> > --- >> > >> > Server certificate >> > >> > -----BEGIN CERTIFICATE----- >> > >> > ... >> > >> > -----END CERTIFICATE----- >> > >> > ... >> > >> > --- >> > >> > Acceptable client certificate CA names >> > >> > --- >> > >> > SSL handshake has read 2017 bytes and written 4534 bytes >> > >> > --- >> > >> > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384 >> > >> > Server public key is 2048 bit >> > >> > Secure Renegotiation IS supported >> > >> > Compression: NONE >> > >> > Expansion: NONE >> > >> > SSL-Session: >> > >> > Protocol : TLSv1.2 >> > >> > Cipher : ECDHE-RSA-AES256-GCM-SHA384 >> > >> > Session-ID: >> > 5A09AF66C19A54A200221F9EFACC20642DBDCCE50099EE6836FDA0B4ECE33EF6 >> > >> > Session-ID-ctx: >> > >> > Master-Key: >> > F4BD9CEA03E292AC4DC696B46E3CD1348BD954C300FAE6A07697507937B4 >> > 22187B51FB0814B20CFBCAFD21A65B30BEBC >> > >> > >> > Key-Arg : None >> > >> > Krb5 Principal: None >> > >> > PSK identity: None >> > >> > PSK identity hint: None >> > >> > Start Time: 1510584166 >> > >> > Timeout : 300 (sec) >> > >> > * Verify return code: 0 (ok) * >> > >> > --- >> > >> > -bash-4.1$ >> > >> > -- >> > >> > *Sandeep Kumar,* >> > Mobile +91-9866507368 <+91%2098665%2007368> >> > >> > *“Happiness is not a destination, It is the journey”* >> > >> > -- > > *Sandeep Kumar,* > Mobile +91-9866507368 <+91%2098665%2007368> > > *“Happiness is not a destination, It is the journey”* > > -- *Sandeep Kumar,* Mobile +91-9866507368 *“Happiness is not a destination, It is the journey”*
