Hi All, For the record – these instructions are spot on and worked like a charm – thanks Matt. Conrad
From: Matthew Clarke <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, 15 February 2016 at 17:12 To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: SSL configuration on cluster The communications between secure NiFi cluster Nodes uses 2-way SSL which requires both keystores and truststores. The truststore typically includes a bunch of Certificate Authorities, but in your case since you are using self-signed certificates for each of your nodes, it will also need to include the keys for everyone of your nodes. You can simply this by creating your own CA to sign all certificates. Then you only need to add your CA to each nodes truststore. You can use a site tinycert.org<http://tinycert.org> top create yoru won CA and certificates for free or the following procedure may be helpful if you want to create them locally yourself: You only need to create one Certificate Authority (CA), which you will use to sign the keys for every one of your servers/VMs and users (You only need to create keys for users if your NiFi has not been configured to use LDAP authentication). What is a CA? The CA acts as a trusted entity for validating the authenticity of certificates. Commands for creating a CA: 1. openssl genrsa -out rootCA.key 2048 2. openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out rootCA.pem 3. openssl x509 -outform der -in rootCA.pem -out rootCA.der 4. keytool -import -keystore cacerts.jks -file rootCA.der At the end of the above you will have your "truststore" file (cacerts.jks) that you will use in your nifi.properties file. Use this same "truststore" file on every one of your servers/VMs. Now lets create a server/vm key and get it signed by that CA: These are the steps I follow: 1. keytool -genkey -alias nifi-server1 -keyalg RSA -keystore nifi-server1.jks -keysize 2048 2. keytool -certreq -alias nifi-server1 -keystore nifi-server1.jks -file nifi-server1.csr 3. openssl x509 -req -in nifi-server1.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out nifi-server1.crt -days 1000 4. keytool -import -keystore nifi-server1.jks -file rootCA.pem 5. keytool -import -trustcacerts -alias nifi-server1 -file nifi-server1.crt -keystore nifi-server1.jks At the end of the above you will have your "keystore" file (nifi-server1.jks) that you will use in your nifi.properties file for one of your servers/VMs. You will need to repeat the above steps for each of your other servers/VMs so they each use their own keys. Now keep in mind that I am using “nifi-server1" in this example, but you will most likely use your systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly recommend that you use the same key and keystore password for every key you create if creating keys for multiple nodes in a NiFi cluster. On Mon, Feb 15, 2016 at 11:39 AM, Conrad Crampton <[email protected]<mailto:[email protected]>> wrote: Hi, New to this list and loving NiFi so far ;-) I have set up and been trying out with good results a NiFi set up locally on my laptop – all good. Managed to set up SSL configuration and hook up with company LDAP. I have also set up three node cluster with cluster manager on a separate host, all using http – all good here too. NCM communicates fine with three nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP is where I have come a bit unstuck. I am using self signed certificate – on laptop used one generated using openssl, then converted to pks file (which worked fine as above). The issue I’m having is running virtually the same setup on cluster as laptop fails with the following exception (there is loads of other nested exception details) but it appears to boil down to this one. Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'protocolSocketConfiguration': FactoryBean threw exception on object creation; nested exception is java.security.KeyStoreException: not found at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175) ~[na:na] at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) ~[na:na] at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517) ~[na:na] at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314) ~[na:na] at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[na:na] at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) ~[na:na] ... 103 common frames omitted Caused by: java.security.KeyStoreException: not found at java.security.KeyStore.getInstance(KeyStore.java:851) ~[na:1.8.0_51] at org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72) ~[na:na] at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46) ~[na:na] at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31) ~[na:na] at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) ~[na:na] ... 108 common frames omitted Caused by: java.security.NoSuchAlgorithmException: KeyStore not available at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) ~[na:1.8.0_51] at java.security.Security.getImpl(Security.java:695) ~[na:1.8.0_51] at java.security.KeyStore.getInstance(KeyStore.java:848) ~[na:1.8.0_51] Having looked at the code, line 72 (&73) of SSLContextFactory is final KeyStore trustStore = KeyStore.getInstance(truststoreType); final FileInputStream trustStoreStream = new FileInputStream(truststore); Which is throwing the error by the looks of it, so appears to need a value in the nifi.properties file for nifi.security.truststoreType= which on my laptop config is blank (docs seems to suggest that this is optional anyway), but the code complains about this not being present. If I enter a value (JKS or PKCS12) in here I get a filenotfound exception on line 73 instead as I haven’t provided a value for truststore location. So, do I need to provide truststore configuration in a clustered environment? If so, docs would helpful to point this out, if not anyone suggest something to fix this please? It took me 10 mins to set up on laptop, now after 6 hours of trying on cluster I’m stumped. Thanks Conrad SecureData, combating cyber threats ________________________________ The information contained in this message or any of its attachments may be privileged and confidential and intended for the exclusive use of the intended recipient. If you are not the intended recipient any disclosure, reproduction, distribution or other dissemination or use of this communications is strictly prohibited. The views expressed in this email are those of the individual and not necessarily of SecureData Europe Ltd. Any prices quoted are only valid if followed up by a formal written quote. SecureData Europe Limited. Registered in England & Wales 04365896. Registered Address: SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent, ME16 9NT ***This email originated outside SecureData*** Click here<https://www.mailcontrol.com/sr/Ni4e5Ve2jFPGX2PQPOmvUhkLFoJbzkFEUGpOKun7w!QfrdtWVdOniSZyTuGcCNTxpHEUYPvxwk+TKOxjbsImtQ==> to report this email as spam.
