Hi Ricky, Sorry, should have noted that the debug output goes to nifi-bootstrap.log, so thanks Mark for jumping in to help there.
If you look at the top of that log, you’ll note that there is no keystore file provided and the truststore loaded is the default JRE cacerts truststore. Can you please provide your nifi.properties file in a Gist, *taking care to redact any sensitive values* like keystore/truststore passwords, although I think from looking at your log output, you are taking advantage of the encrypted configuration feature, so even viewing the encrypted values should be ok. Could you also please provide the directory listing where the keystore and truststore are located including the permissions and ownership information? There may be a bug in the logic between cluster and standalone mode, but I haven’t encountered this behavior before. If you can start NiFi in standalone mode, could you please provide the output of the following command run from the terminal? It will simulate an HTTPS connection to the server and verify the key and certificate presented by NiFi. * host — the NiFi hostname * port — the port NiFi is running on * path_to_your_cert.pem — the public key certificate identifying the client/user (i.e. what you load into your browser to authenticate) * path_to_your_key.key — the private key identifying the client/user * path_to_your_CA_cert.pem — the public key certificate identifying the CA which signed your NiFi server certificate (if self-signed, provide that certificate) $ openssl s_client -connect <host:port> -debug -state -cert <path_to_your_cert.pem> -key <path_to_your_key.pem> -CAfile <path_to_your_CA_cert.pem> Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Nov 4, 2016, at 11:21 AM, Ricky Saltzer <[email protected]> wrote: > > Hey guys - > > I went ahead and uploaded the boostrap log. I took a look at it and it > seems to be the same error [1] > > [1]: > https://gist.githubusercontent.com/rickysaltzer/b156594f92066873f80928eddf84e7bb/raw/4d0e018038b332f4fdf14644910dfa9e70c57e49/gistfile1.txt > > On Fri, Nov 4, 2016 at 2:14 PM, Mark Payne <[email protected]> wrote: > >> Hey Ricky, >> >> When you enable debug logging for SSL, it writes to StdErr (or StdOut?) so >> it will end up in your logs/nifi-bootstrap.log instead of nifi-app.log. >> Can you give that a look? >> >> Thanks >> -Mark >> >>> On Nov 4, 2016, at 2:07 PM, Ricky Saltzer <[email protected]> wrote: >>> >>> Hey Andy - >>> >>> Thanks for the response. I'm currently just trying to get one node in >>> clustered mode before adding a second. The keystore is stored locally and >>> I've confirmed it's readable, as it was able to start once I took it out >> of >>> clustered mode. I added that line to the bootstrap.conf, but I don't >>> believe any additional logging was produced in regards to troubleshooting >>> this problem. Just in case, I've attached the entire log [1]. >>> >>> [1]: >>> https://gist.githubusercontent.com/rickysaltzer/ >> ed454d87d2207d5acab401a473d4be57/raw/425c42da762fc5cc997153d48b09f0 >> fedabc88bb/gistfile1.txt <https://gist.githubusercontent.com/rickysaltzer/ >> ed454d87d2207d5acab401a473d4be57/raw/425c42da762fc5cc997153d48b09f0 >> fedabc88bb/gistfile1.txt> >>> >>> On Wed, Nov 2, 2016 at 7:08 PM, Andy LoPresto <[email protected] >> <mailto:[email protected]>> wrote: >>> >>>> Hi Ricky, >>>> >>>> Sorry to hear you are having this issue. Is the keystore available on >> all >>>> nodes of the cluster? It appears from the log message that the keystore >> is >>>> not found during startup. To further debug, you can add the following >> line >>>> in bootstrap.conf to provide additional logging: >>>> >>>> java.arg.15=-Djavax.net.debug=ssl,handshake >>>> >>>> Andy LoPresto >>>> [email protected] <mailto:[email protected]> >>>> *[email protected] <mailto:[email protected]> < >> [email protected] <mailto:[email protected]>>* >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>> >>>> On Nov 2, 2016, at 2:25 PM, Ricky Saltzer <[email protected]> wrote: >>>> >>>> Hey all - >>>> >>>> I'm using NiFi 1.0 and I'm having an issue using secure mode with a >> local >>>> key store while in clustered mode. If I set the node in clustered mode, >> and >>>> also provide a valid keystore, I receive a KeyStoreException [1]. If I >> set >>>> the configuration to not use clustered mode, NiFi will start up fine >> with >>>> the provided key store. Am I supposed to be storing this key store in >>>> Zookeeper somewhere? >>>> >>>> >>>> [1] >>>> >>>> >>>> Caused by: java.security.KeyStoreException: not found >>>> >>>> >>>> at java.security.KeyStore.getInstance(KeyStore.java:839) >>>> ~[na:1.8.0_11] >>>> >>>> at >>>> org.apache.nifi.io.socket.SSLContextFactory.<init>( >>>> SSLContextFactory.java:61) >>>> ~[nifi-socket-utils-1.0.0.jar:1.0.0] >>>> >>>> at >>>> org.apache.nifi.cluster.protocol.spring.ServerSocketConfigurationFacto >>>> ryBean.getObject(ServerSocketConfigurationFactoryBean.java:45) >>>> ~[nifi-framework-cluster-protocol-1.0.0.jar:1.0.0] >>>> >>>> at >>>> org.apache.nifi.cluster.protocol.spring.ServerSocketConfigurationFacto >>>> ryBean.getObject(ServerSocketConfigurationFactoryBean.java:30) >>>> ~[nifi-framework-cluster-protocol-1.0.0.jar:1.0.0] >>>> >>>> at >>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport. >>>> doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) >>>> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] >>>> >>>> ... 69 common frames omitted >>>> >>>> Caused by: java.security.NoSuchAlgorithmException: KeyStore not >> available >>>> >>>> at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) >>>> ~[na:1.8.0_11] >>>> >>>> at java.security.Security.getImpl(Security.java:695) >> ~[na:1.8.0_11] >>>> >>>> at java.security.KeyStore.getInstance(KeyStore.java:836) >>>> ~[na:1.8.0_11] >>>> >>>> ... 73 common frames omitted >>>> >>>> >>>> >>> >>> >>> -- >>> Ricky Saltzer >>> http://www.cloudera.com <http://www.cloudera.com/> >> > > > > -- > Ricky Saltzer > http://www.cloudera.com
signature.asc
Description: Message signed with OpenPGP using GPGMail
