Could you post your nifi.properties for each node? (replacing anything sensitive with placeholders)
On Fri, Mar 17, 2017 at 8:46 AM, Ryan H <[email protected]> wrote: > Hi Andy, > > Here is what I have (still stuck): > > * *Using keytool, ensure there are no Extended Key Usage restrictions on > your certificates which prevent them from being used for client > authentication* > > -not really sure how to check this. Everything was generated using the > nifi toolkit. This is what I have from inspecting the client cert > (excluding other information): > > X509v3 Key Usage: critical > > Digital Signature, Non Repudiation, Key Encipherment, Data > Encipherment, Key Agreement > > X509v3 Basic Constraints: > > CA:FALSE > > X509v3 Extended Key Usage: > > TLS Web Client Authentication, TLS Web Server > Authentication > > ** Ensure the Initial Admin Identity is correctly set in authorizers.xml* > > -This is correct (or at least I think it is). In the nifi-user.log, it > shows that the authentication succeeded for the initial admin identity, and > then failed for the following error: > > 2017-03-16 12:39:41,110 INFO [NiFi Web Server-117] > o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: > Kerberos ticket login not supported by this NiFi.. Returning Conflict > response. > > 2017-03-16 12:39:41,280 INFO [NiFi Web Server-18] > o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=adminuser, > OU=NIFI) GET https://my-server-ip-address/nifi-api/flow/current-user (source > ip: my-local-ip-address) > > 2017-03-16 12:39:41,280 INFO [NiFi Web Server-18] > o.a.n.w.s.NiFiAuthenticationFilter Authentication success for > CN=adminuser, OU=NIFI > > ** Check the value of nifi.security.needClientAuth in nifi.properties on > both nodes* > > -There is not a value set in either nifi.properties file. > > > ** Run the following s_client command from one node to connect to the > other, providing the relevant certificates and keys: $ openssl s_client > -connect <host:port> -debug -state -cert <path_to_your_cert.pem> -key > <path_to_your_key.pem> -CAfile <path_to_your_CA_cert.pem>* > > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-SHA256 > > Server public key is 2048 bit > > Secure Renegotiation IS supported > > Compression: NONE > > Expansion: NONE > > SSL-Session: > > Protocol : TLSv1.2 > > Cipher : ECDHE-RSA-AES128-SHA256 > > Session-ID: a long number > > Session-ID-ctx: > > Master-Key: a long number > > Key-Arg : None > > Krb5 Principal: None > > PSK identity: None > > PSK identity hint: None > > Start Time: 1489713403 > > Timeout : 300 (sec) > > Verify return code: 0 (ok) > > --- > > ** Check that both nodes are running on Java 8 and using a recent version > of OpenSSL to allow for compatible cipher suite negotiation* > > *-Java Information:* > *VM-1* > > openjdk version "1.8.0_121" > > OpenJDK Runtime Environment (build 1.8.0_121-b13) > > OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) > > *VM-2* > > java version "1.8.0_121" > > Java(TM) SE Runtime Environment (build 1.8.0_121-b13) > > Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) > > > *OpenSSL Version:* > > OpenSSL 1.0.1e-fips 11 Feb 2013 > > ** Check to see that the Oracle JCE Unlimited Strength Cryptographic > Policy files are installed (only necessary if the only available cipher > suites require 256-bit encryption)* > -This is installed on both servers. > > > *Errors when trying to access the UI are the same. Here are the errors > from the logs:* > > nifi-bootstrap.log > > 2017-03-17 12:37:56,557 INFO [NiFi logging handler] org.apache.nifi.StdOut > NiFi Web Server-19, WRITE: TLSv1.2 Handshake, length = 40 > > 2017-03-17 12:37:56,557 INFO [NiFi logging handler] org.apache.nifi.StdOut > %% Cached server session: [Session-575, TLS_ECDHE_RSA_WITH_AES_256_ > GCM_SHA384] > > 2017-03-17 12:37:56,599 INFO [NiFi logging handler] org.apache.nifi.StdOut > NiFi Web Server-16, called closeInbound() > > 2017-03-17 12:37:56,599 INFO [NiFi logging handler] org.apache.nifi.StdOut > NiFi Web Server-16, fatal error: 80: Inbound closed before receiving peer's > close_notify: possible truncation attack? > > 2017-03-17 12:37:56,599 INFO [NiFi logging handler] org.apache.nifi.StdOut > javax.net.ssl.SSLException: Inbound closed before receiving peer's > close_notify: possible truncation attack? > > 2017-03-17 12:37:56,599 INFO [NiFi logging handler] org.apache.nifi.StdOut > %% Invalidated: [Session-575, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384] > > 2017-03-17 12:37:56,599 INFO [NiFi logging handler] org.apache.nifi.StdOut > NiFi Web Server-16, SEND TLSv1.2 ALERT: fatal, description = internal_error > > 2017-03-17 12:37:56,599 INFO [NiFi logging handler] org.apache.nifi.StdOut > NiFi Web Server-16, WRITE: TLSv1.2 Alert, length = 26 > > 2017-03-17 12:37:56,601 INFO [NiFi logging handler] org.apache.nifi.StdOut > NiFi Web Server-15, WRITE: TLSv1.2 Application Data, length = 1356 > > > nifi-app.log > > 2017-03-17 12:44:00,059 WARN [Replicate Request Thread-7] > o.a.n.c.c.h.r.ThreadPoolRequestReplicator > Failed to replicate request GET /nifi-api/flow/current-user to > ip-10-227-80-39:443 due to {} > > com.sun.jersey.api.client.ClientHandlerException: > javax.net.ssl.SSLHandshakeException: > Remote host closed connection during handshake > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.Client.handle(Client.java:652) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle( > GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) > ~[jersey-client-1.19.jar:1.19] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578) > ~[nifi-framework-cluster-1.1.1.jar:1.1.1] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator$NodeHttpRequest.run( > ThreadPoolRequestReplicator.java:770) ~[nifi-framework-cluster-1.1. > 1.jar:1.1.1] > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_121] > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_121] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] > > Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed > connection during handshake > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992) > ~[na:1.8.0_121] > > at > sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect( > AbstractDelegateHttpsURLConnection.java:185) ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) > ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) > ~[na:1.8.0_121] > > at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode( > HttpsURLConnectionImpl.java:338) ~[na:1.8.0_121] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke( > URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19] > > ... 12 common frames omitted > > Caused by: java.io.EOFException: SSL peer shut down incorrectly > > at sun.security.ssl.InputRecord.read(InputRecord.java:505) ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) > ~[na:1.8.0_121] > > ... 23 common frames omitted > > 2017-03-17 12:44:00,062 WARN [Replicate Request Thread-7] o.a.n.c.c.h.r. > ThreadPoolRequestReplicator > > com.sun.jersey.api.client.ClientHandlerException: > javax.net.ssl.SSLHandshakeException: > Remote host closed connection during handshake > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.Client.handle(Client.java:652) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle( > GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) > ~[jersey-client-1.19.jar:1.19] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578) > ~[nifi-framework-cluster-1.1.1.jar:1.1.1] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator$NodeHttpRequest.run( > ThreadPoolRequestReplicator.java:770) ~[nifi-framework-cluster-1.1. > 1.jar:1.1.1] > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_121] > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_121] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] > > Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed > connection during handshake > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992) > ~[na:1.8.0_121] > > at > sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect( > AbstractDelegateHttpsURLConnection.java:185) ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) > ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) > ~[na:1.8.0_121] > > at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode( > HttpsURLConnectionImpl.java:338) ~[na:1.8.0_121] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke( > URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19] > > ... 12 common frames omitted > > Caused by: java.io.EOFException: SSL peer shut down incorrectly > > at sun.security.ssl.InputRecord.read(InputRecord.java:505) ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) > ~[na:1.8.0_121] > > ... 23 common frames omitted > > 2017-03-17 12:44:00,071 WARN [Replicate Request Thread-8] > o.a.n.c.c.h.r.ThreadPoolRequestReplicator > Failed to replicate request GET /nifi-api/flow/current-user to > my-server-hostname:443 due to {} > > com.sun.jersey.api.client.ClientHandlerException: > javax.net.ssl.SSLHandshakeException: > Remote host closed connection during handshake > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.Client.handle(Client.java:652) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle( > GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) > ~[jersey-client-1.19.jar:1.19] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578) > ~[nifi-framework-cluster-1.1.1.jar:1.1.1] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator$NodeHttpRequest.run( > ThreadPoolRequestReplicator.java:770) ~[nifi-framework-cluster-1.1. > 1.jar:1.1.1] > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_121] > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_121] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] > > Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed > connection during handshake > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992) > ~[na:1.8.0_121] > > at > sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect( > AbstractDelegateHttpsURLConnection.java:185) ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) > ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) > ~[na:1.8.0_121] > > at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode( > HttpsURLConnectionImpl.java:338) ~[na:1.8.0_121] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke( > URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19] > > ... 12 common frames omitted > > Caused by: java.io.EOFException: SSL peer shut down incorrectly > > at sun.security.ssl.InputRecord.read(InputRecord.java:505) ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) > ~[na:1.8.0_121] > > ... 23 common frames omitted > > 2017-03-17 12:44:00,074 WARN [Replicate Request Thread-8] o.a.n.c.c.h.r. > ThreadPoolRequestReplicator > > com.sun.jersey.api.client.ClientHandlerException: > javax.net.ssl.SSLHandshakeException: > Remote host closed connection during handshake > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.Client.handle(Client.java:652) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle( > GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) > ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) > ~[jersey-client-1.19.jar:1.19] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578) > ~[nifi-framework-cluster-1.1.1.jar:1.1.1] > > at org.apache.nifi.cluster.coordination.http.replication. > ThreadPoolRequestReplicator$NodeHttpRequest.run( > ThreadPoolRequestReplicator.java:770) ~[nifi-framework-cluster-1.1. > 1.jar:1.1.1] > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_121] > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_121] > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_121] > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] > > Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed > connection during handshake > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992) > ~[na:1.8.0_121] > > at > sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) > ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect( > AbstractDelegateHttpsURLConnection.java:185) ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) > ~[na:1.8.0_121] > > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) > ~[na:1.8.0_121] > > at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) > ~[na:1.8.0_121] > > at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode( > HttpsURLConnectionImpl.java:338) ~[na:1.8.0_121] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke( > URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19] > > at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle( > URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19] > > ... 12 common frames omitted > > Caused by: java.io.EOFException: SSL peer shut down incorrectly > > at sun.security.ssl.InputRecord.read(InputRecord.java:505) ~[na:1.8.0_121] > > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) > ~[na:1.8.0_121] > > ... 23 common frames omitted > > nifi-user.log > > 2017-03-17 12:43:59,834 INFO [NiFi Web Server-15] > o.a.n.w.a.c.IllegalStateExceptionMapper > java.lang.IllegalStateException: Kerberos ticket login not supported by > this NiFi.. Returning Conflict response. > 2017-03-17 12:44:00,052 INFO [NiFi Web Server-125] > o.a.n.w.s.NiFiAuthenticationFilter > Attempting request for (CN=adminuser, OU=NIFI) GET > https://my-server-address/nifi-api/flow/current-user (source ip: > my-local-address) > 2017-03-17 12:44:00,052 INFO [NiFi Web Server-125] > o.a.n.w.s.NiFiAuthenticationFilter > Authentication success for CN=adminuser, OU=NIFI > > On Thu, Mar 16, 2017 at 2:54 PM, Andy LoPresto <[email protected]> > wrote: > >> Hi Ryan, >> >> Sorry to hear you are having issues. Can you please check the following >> things? >> >> * Using keytool, ensure there are no Extended Key Usage restrictions on >> your certificates which prevent them from being used for client >> authentication >> * Ensure the Initial Admin Identity is correctly set in authorizers.xml >> * Check the value of nifi.security.needClientAuth in nifi.properties on >> both nodes >> * Run the following s_client command from one node to connect to the >> other, providing the relevant certificates and keys: $ openssl s_client >> -connect <host:port> -debug -state -cert <path_to_your_cert.pem> -key >> <path_to_your_key.pem> -CAfile <path_to_your_CA_cert.pem> >> * Check that both nodes are running on Java 8 and using a recent version >> of OpenSSL to allow for compatible cipher suite negotiation >> * Check to see that the Oracle JCE Unlimited Strength Cryptographic >> Policy files are installed (only necessary if the only available cipher >> suites require 256-bit encryption) >> >> Andy LoPresto >> [email protected] >> *[email protected] <[email protected]>* >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On Mar 16, 2017, at 6:04 AM, Ryan H <[email protected]> >> wrote: >> >> Hi All, >> >> I am working on setting up a 2 node secure NiFi Cluster (version 1.1.1). >> I am using the NiFi Toolkit in standalone mode for the cert and other >> information. The 2 nodes are on AWS EC2 instances (1 node per EC2 >> instance). I am using and embedded Zookeeper for each instance for now, and >> will move to use external Zookeeper (3 instances) once I have worked out >> the following setup. >> >> There do not look to be any errors during startup. As soon as making the >> request from my web browser (with the client cert created from the >> Toolkit), It results in an error. After that all looks well in the logs >> with regard to recurring errors. >> >> The error displayed in the UI is: "An unexpected error has occurred, >> please check the logs" >> >> Searching thru the logs I am seeing the following errors: >> >> *nifi-app.log* >> >> 2017-03-16 12:58:44,199 WARN [Replicate Request Thread-5] >> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request >> GET /nifi-api/flow/current-user to my-server-hostname:443 due to {} >> >> com.sun.jersey.api.client.ClientHandlerException: >> javax.net.ssl.SSLHandshakeException: Remote host closed connection >> during handshake >> >> at com.sun.jersey.client.urlconnection.URLConnectionClientHandl >> er.handle(URLConnectionClientHandler.java:155) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.Client.handle(Client.java:652) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.h >> andle(GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) >> ~[jersey-client-1.19.jar:1.19] >> >> at org.apache.nifi.cluster.coordination.http.replication.Thread >> PoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578) >> ~[nifi-framework-cluster-1.1.1.jar:1.1.1] >> >> at org.apache.nifi.cluster.coordination.http.replication.Thread >> PoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:770) >> ~[nifi-framework-cluster-1.1.1.jar:1.1.1] >> >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> [na:1.8.0_121] >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> [na:1.8.0_121] >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> [na:1.8.0_121] >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> [na:1.8.0_121] >> >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] >> >> Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed >> connection during handshake >> >> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992) >> ~[na:1.8.0_121] >> >> at >> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) >> ~[na:1.8.0_121] >> >> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) >> ~[na:1.8.0_121] >> >> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) >> ~[na:1.8.0_121] >> >> at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) >> ~[na:1.8.0_121] >> >> at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnectio >> n.connect(AbstractDelegateHttpsURLConnection.java:185) ~[na:1.8.0_121] >> >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) >> ~[na:1.8.0_121] >> >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) >> ~[na:1.8.0_121] >> >> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) >> ~[na:1.8.0_121] >> >> at sun.net.www.protocol.https.HttpsURLConnectionImpl.getRespons >> eCode(HttpsURLConnectionImpl.java:338) ~[na:1.8.0_121] >> >> at com.sun.jersey.client.urlconnection.URLConnectionClientHandl >> er._invoke(URLConnectionClientHandler.java:253) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.client.urlconnection.URLConnectionClientHandl >> er.handle(URLConnectionClientHandler.java:153) >> ~[jersey-client-1.19.jar:1.19] >> >> ... 12 common frames omitted >> >> Caused by: java.io.EOFException: SSL peer shut down incorrectly >> >> at sun.security.ssl.InputRecord.read(InputRecord.java:505) >> ~[na:1.8.0_121] >> >> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) >> ~[na:1.8.0_121] >> >> ... 23 common frames omitted >> >> 2017-03-16 12:58:44,200 WARN [Replicate Request Thread-5] >> o.a.n.c.c.h.r.ThreadPoolRequestReplicator >> >> com.sun.jersey.api.client.ClientHandlerException: >> javax.net.ssl.SSLHandshakeException: Remote host closed connection >> during handshake >> >> at com.sun.jersey.client.urlconnection.URLConnectionClientHandl >> er.handle(URLConnectionClientHandler.java:155) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.Client.handle(Client.java:652) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.h >> andle(GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) >> ~[jersey-client-1.19.jar:1.19] >> >> at org.apache.nifi.cluster.coordination.http.replication.Thread >> PoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578) >> ~[nifi-framework-cluster-1.1.1.jar:1.1.1] >> >> at org.apache.nifi.cluster.coordination.http.replication.Thread >> PoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:770) >> ~[nifi-framework-cluster-1.1.1.jar:1.1.1] >> >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> [na:1.8.0_121] >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> [na:1.8.0_121] >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> [na:1.8.0_121] >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> [na:1.8.0_121] >> >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] >> >> Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed >> connection during handshake >> >> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992) >> ~[na:1.8.0_121] >> >> at >> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) >> ~[na:1.8.0_121] >> >> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) >> ~[na:1.8.0_121] >> >> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) >> ~[na:1.8.0_121] >> >> at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) >> ~[na:1.8.0_121] >> >> at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnectio >> n.connect(AbstractDelegateHttpsURLConnection.java:185) ~[na:1.8.0_121] >> >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) >> ~[na:1.8.0_121] >> >> at >> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) >> ~[na:1.8.0_121] >> >> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) >> ~[na:1.8.0_121] >> >> at sun.net.www.protocol.https.HttpsURLConnectionImpl.getRespons >> eCode(HttpsURLConnectionImpl.java:338) ~[na:1.8.0_121] >> >> at com.sun.jersey.client.urlconnection.URLConnectionClientHandl >> er._invoke(URLConnectionClientHandler.java:253) >> ~[jersey-client-1.19.jar:1.19] >> >> at com.sun.jersey.client.urlconnection.URLConnectionClientHandl >> er.handle(URLConnectionClientHandler.java:153) >> ~[jersey-client-1.19.jar:1.19] >> >> ... 12 common frames omitted >> >> Caused by: java.io.EOFException: SSL peer shut down incorrectly >> >> at sun.security.ssl.InputRecord.read(InputRecord.java:505) >> ~[na:1.8.0_121] >> >> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) >> ~[na:1.8.0_121] >> >> ... 23 common frames omitted >> >> *nifi-bootstrap.log (additional logging turned on in bootstrap.config)* >> >> 2017-03-16 12:39:41,195 INFO [NiFi logging handler] >> org.apache.nifi.StdOut *** ServerHelloDone >> >> 2017-03-16 12:39:41,195 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-117, WRITE: TLSv1.2 Handshake, >> length = 2225 >> >> 2017-03-16 12:39:41,196 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-16, WRITE: TLSv1.2 Application Data, >> length = 16384 >> >> 2017-03-16 12:39:41,197 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-16, WRITE: TLSv1.2 Application Data, >> length = 5589 >> >> 2017-03-16 12:39:41,197 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-16, WRITE: TLSv1.2 Application Data, >> length = 7213 >> >> 2017-03-16 12:39:41,197 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-16, WRITE: TLSv1.2 Application Data, >> length = 7 >> >> 2017-03-16 12:39:41,270 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-122, WRITE: TLSv1.2 Application >> Data, length = 16384 >> >> 2017-03-16 12:39:41,270 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-122, WRITE: TLSv1.2 Application >> Data, length = 5378 >> >> 2017-03-16 12:39:41,272 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-122, WRITE: TLSv1.2 Application >> Data, length = 16384 >> >> 2017-03-16 12:39:41,275 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-118, called closeInbound() >> >> *2017-03-16 12:39:41,275 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-118, fatal error: 80: Inbound closed >> before receiving peer's close_notify: possible truncation attack?* >> >> *2017-03-16 12:39:41,275 INFO [NiFi logging handler] >> org.apache.nifi.StdOut javax.net.ssl.SSLException: Inbound closed before >> receiving peer's close_notify: possible truncation attack?* >> >> 2017-03-16 12:39:41,275 INFO [NiFi logging handler] >> org.apache.nifi.StdOut %% Invalidated: [Session-7405, >> TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384] >> >> 2017-03-16 12:39:41,275 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-118, SEND TLSv1.2 ALERT: fatal, >> description = internal_error >> >> 2017-03-16 12:39:41,275 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-118, WRITE: TLSv1.2 Alert, length = 2 >> >> 2017-03-16 12:39:41,282 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-117, WRITE: TLSv1.2 Application >> Data, length = 7663 >> >> 2017-03-16 12:39:41,282 INFO [NiFi logging handler] >> org.apache.nifi.StdOut NiFi Web Server-117, WRITE: TLSv1.2 Application >> Data, length = 7 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Allow unsafe renegotiation: false >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Allow legacy hello messages: true >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Is initial handshake: true >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Is secure renegotiation: false >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Replicate Request Thread-4, setSoTimeout(5000) called >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring disabled protocol: SSLv3 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1 >> >> 2017-03-16 12:39:41,289 INFO [NiFi logging handler] >> org.apache.nifi.StdOut Ignoring unsupported cipher suite: >> TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1.1 >> >> 2017-03-16 12:39:41,290 INFO [NiFi logging handler] >> org.apache.nifi.StdOut %% No cached client session >> >> 2017-03-16 12:39:41,290 INFO [NiFi logging handler] >> org.apache.nifi.StdOut *** ClientHello, TLSv1.2 >> >> >> *nifi-user.log* >> >> 2017-03-16 12:39:41,110 INFO [NiFi Web Server-117] >> o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: >> Kerberos ticket login not supported by this NiFi.. Returning Conflict >> response. >> >> 2017-03-16 12:39:41,280 INFO [NiFi Web Server-18] >> o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=adminuser, >> OU=NIFI) GET https://my-server-ip-address/nifi-api/flow/current-user >> (source ip: my-local-ip-address) >> >> 2017-03-16 12:39:41,280 INFO [NiFi Web Server-18] >> o.a.n.w.s.NiFiAuthenticationFilter Authentication success for >> CN=adminuser, OU=NIFI >> >> >> Main articles followed for the secure cluster setup and for points of >> reference: >> >> http://bryanbende.com/development/2016/08/17/apache-nifi-1- >> 0-0-authorization-and-multi-tenancy >> >> https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-secur >> ed-cluster-setup/ >> >> https://nifi.apache.org/docs/nifi-docs/html/administration-g >> uide.html#security-configuration >> >> >> Any help or direction is greatly appreciated! >> >> >> Cheers, >> >> Ryan H. >> >> >> >
