Re: KnoxSSO with NiFi error: PKIX path building failed...
Ryan, Sure, you can reply with the content here. On Fri, Mar 9, 2018 at 9:24 AM Ryan H wrote: > Hi Jeff, > > I created txt files with the output for the ssl-debug output, but unable > to attach a file to this email. I can paste it into the email if need be. > Let me know if that is what you prefer or if I should do something else. > > -Ryan > > On Thu, Mar 8, 2018 at 11:59 PM, Jeff wrote: > >> Ryan, >> >> I just subscribed to the user list here, but haven't pulled the previous >> messages. I did read your responses on the public archive. It's >> interesting that you're getting the SSLPeerUnverifiedException. You >> could try turning on SSL debug in Knox and taking a look (and/or attaching >> that log information to this thread). >> >> Could you provide information on how you used the TLS Toolkit to generate >> the key/truststore? If you can do a verbose listing of the keystore with >> keytool and capture the output to provide in this thread, that would be >> helpful as well. There are a few people on the NiFi team that are more >> knowledgable about the TLS Toolkit and SSL than I am with which I could >> consult. >> >> On Thu, Mar 8, 2018 at 12:49 PM Jeff wrote: >> >>> Ryan, >>> >>> In addition to the things I mentioned in my previous message, there are >>> some properties you'll need to set as well. In nifi.properties, set >>> nifi.web.proxy.context.path to "/gateway/sandbox/nifi-app". The host and >>> port of the Knox service should also be set for nifi.web.proxy.host. >>> >>> On Thu, Mar 8, 2018 at 12:06 PM Jeff wrote: >>> Ryan, I noticed in your sandbox.xml, you have the "useTwoWaySsl" parameter set to false. That needs to be set to true so that the dispatch will create an SSL context that uses the keystore and truststore material in gateway.jks. I'm glad you're using the TLS Toolkit, I was going to suggest you give that a try, initially. The cert from the keystore generated by the toolkit that identifies the cert to use for Knox needs to be added to gateway.jks, along with the nifi-cert key from the truststore. Just importing both the keystore and truststore generated by the toolkit for Knox should be all you have to do there, since the toolkit generates those stores with just the nifi-key and nifi-cert in the keystore and truststore respectively. You should end up with three keys in gateway.jks afterward; thy gateway-identity, nifi-key, and nifi-cert keys. Once both of those are added to gateway.jks, and you have configured the service definition for NiFi in your topology with useTwoWaySsl set to true, the two-way SSL handshake should succeed. Also, you will want to add the DN from that nifi-key as a node identity (in the same place you set the initial admin identity) so that NiFi can create a "user" to represent the Knox node and add a policy for you to allow that node/identity to proxy requests, if you haven't already done so. After adding the keystore and truststore material to gateway.jks, added a user and policy for NiFi to identify and authorize Knox for proxying, Knox should be able to proxy NiFi securely. On Thu, Mar 8, 2018 at 11:44 AM larry mccay wrote: > There was definitely some nuances to this area and you need to be > careful with confusing the truststore for Knox accepting client certs and > Knox sending client certs from dispatch to the back end services. You may > find better luck adding the nifi server cert or its ca cert to the Knox > machine cacerts truststore. > > I am explicitly adding @jeff here as well since he is the Nifi > integration guru. > > On Thu, Mar 8, 2018 at 11:09 AM, Ryan H < > ryan.howell.developm...@gmail.com> wrote: > >> Ok. Well I've imported the truststore from NiFi into the gateway >> keystore but I am still unable to connect to NiFi via Knox with the same >> error. Password for the NiFi truststore is the same as the gateway >> keystore >> just to make sure that wasn't an issue. I've changed some of the >> gateway-site.xml settings to explicitly trust certificates and to use the >> gateway.jks store, but still getting the same error. It's clearly an ssl >> error between Knox and NiFi. I'll just keep digging I suppose until it >> gets >> worked out. Not really sure what else to try here unfortunately... >> >> -Ryan >> >> On Thu, Mar 8, 2018 at 9:23 AM, Sandeep Moré >> wrote: >> >>> I am not that aware of NiFi specific SSL settings, but if you have a >>> truststore from NiFi (instead of a cert) you can use that, if it has a >>> deifferent password you will have to configure it. You can find the >>> instructions here >>> https://knox.apache.org/books/knox-1-0-0/user-guide.html#Mutual+Authentication+with+SSL >>> >>> >>> >>> On Wed, Mar 7, 2018 at
Re: KnoxSSO with NiFi error: PKIX path building failed...
Hi Jeff, I created txt files with the output for the ssl-debug output, but unable to attach a file to this email. I can paste it into the email if need be. Let me know if that is what you prefer or if I should do something else. -Ryan On Thu, Mar 8, 2018 at 11:59 PM, Jeff wrote: > Ryan, > > I just subscribed to the user list here, but haven't pulled the previous > messages. I did read your responses on the public archive. It's > interesting that you're getting the SSLPeerUnverifiedException. You could > try turning on SSL debug in Knox and taking a look (and/or attaching that > log information to this thread). > > Could you provide information on how you used the TLS Toolkit to generate > the key/truststore? If you can do a verbose listing of the keystore with > keytool and capture the output to provide in this thread, that would be > helpful as well. There are a few people on the NiFi team that are more > knowledgable about the TLS Toolkit and SSL than I am with which I could > consult. > > On Thu, Mar 8, 2018 at 12:49 PM Jeff wrote: > >> Ryan, >> >> In addition to the things I mentioned in my previous message, there are >> some properties you'll need to set as well. In nifi.properties, set >> nifi.web.proxy.context.path to "/gateway/sandbox/nifi-app". The host and >> port of the Knox service should also be set for nifi.web.proxy.host. >> >> On Thu, Mar 8, 2018 at 12:06 PM Jeff wrote: >> >>> Ryan, >>> >>> I noticed in your sandbox.xml, you have the "useTwoWaySsl" parameter set >>> to false. That needs to be set to true so that the dispatch will create an >>> SSL context that uses the keystore and truststore material in gateway.jks. >>> >>> I'm glad you're using the TLS Toolkit, I was going to suggest you give >>> that a try, initially. The cert from the keystore generated by the toolkit >>> that identifies the cert to use for Knox needs to be added to gateway.jks, >>> along with the nifi-cert key from the truststore. Just importing both the >>> keystore and truststore generated by the toolkit for Knox should be all you >>> have to do there, since the toolkit generates those stores with just the >>> nifi-key and nifi-cert in the keystore and truststore respectively. You >>> should end up with three keys in gateway.jks afterward; thy >>> gateway-identity, nifi-key, and nifi-cert keys. Once both of those are >>> added to gateway.jks, and you have configured the service definition for >>> NiFi in your topology with useTwoWaySsl set to true, the two-way SSL >>> handshake should succeed. >>> >>> Also, you will want to add the DN from that nifi-key as a node identity >>> (in the same place you set the initial admin identity) so that NiFi can >>> create a "user" to represent the Knox node and add a policy for you to >>> allow that node/identity to proxy requests, if you haven't already done so. >>> >>> After adding the keystore and truststore material to gateway.jks, added >>> a user and policy for NiFi to identify and authorize Knox for proxying, >>> Knox should be able to proxy NiFi securely. >>> >>> On Thu, Mar 8, 2018 at 11:44 AM larry mccay wrote: >>> There was definitely some nuances to this area and you need to be careful with confusing the truststore for Knox accepting client certs and Knox sending client certs from dispatch to the back end services. You may find better luck adding the nifi server cert or its ca cert to the Knox machine cacerts truststore. I am explicitly adding @jeff here as well since he is the Nifi integration guru. On Thu, Mar 8, 2018 at 11:09 AM, Ryan H >>> gmail.com> wrote: > Ok. Well I've imported the truststore from NiFi into the gateway > keystore but I am still unable to connect to NiFi via Knox with the same > error. Password for the NiFi truststore is the same as the gateway > keystore > just to make sure that wasn't an issue. I've changed some of the > gateway-site.xml settings to explicitly trust certificates and to use the > gateway.jks store, but still getting the same error. It's clearly an ssl > error between Knox and NiFi. I'll just keep digging I suppose until it > gets > worked out. Not really sure what else to try here unfortunately... > > -Ryan > > On Thu, Mar 8, 2018 at 9:23 AM, Sandeep Moré > wrote: > >> I am not that aware of NiFi specific SSL settings, but if you have a >> truststore from NiFi (instead of a cert) you can use that, if it has a >> deifferent password you will have to configure it. You can find the >> instructions here https://knox.apache.org/ >> books/knox-1-0-0/user-guide.html#Mutual+Authentication+with+SSL >> >> >> >> On Wed, Mar 7, 2018 at 8:27 PM, Ryan H > gmail.com> wrote: >> >>> Hi Sandeep, >>> >>> So I have the NiFi TLS Toolkit running in Client/Server mode. I have >>> made a request to the CA server from the Knox machine by running the TLS
Re: KnoxSSO with NiFi error: PKIX path building failed...
Ryan, I just subscribed to the user list here, but haven't pulled the previous messages. I did read your responses on the public archive. It's interesting that you're getting the SSLPeerUnverifiedException. You could try turning on SSL debug in Knox and taking a look (and/or attaching that log information to this thread). Could you provide information on how you used the TLS Toolkit to generate the key/truststore? If you can do a verbose listing of the keystore with keytool and capture the output to provide in this thread, that would be helpful as well. There are a few people on the NiFi team that are more knowledgable about the TLS Toolkit and SSL than I am with which I could consult. On Thu, Mar 8, 2018 at 12:49 PM Jeff wrote: > Ryan, > > In addition to the things I mentioned in my previous message, there are > some properties you'll need to set as well. In nifi.properties, set > nifi.web.proxy.context.path to "/gateway/sandbox/nifi-app". The host and > port of the Knox service should also be set for nifi.web.proxy.host. > > On Thu, Mar 8, 2018 at 12:06 PM Jeff wrote: > >> Ryan, >> >> I noticed in your sandbox.xml, you have the "useTwoWaySsl" parameter set >> to false. That needs to be set to true so that the dispatch will create an >> SSL context that uses the keystore and truststore material in gateway.jks. >> >> I'm glad you're using the TLS Toolkit, I was going to suggest you give >> that a try, initially. The cert from the keystore generated by the toolkit >> that identifies the cert to use for Knox needs to be added to gateway.jks, >> along with the nifi-cert key from the truststore. Just importing both the >> keystore and truststore generated by the toolkit for Knox should be all you >> have to do there, since the toolkit generates those stores with just the >> nifi-key and nifi-cert in the keystore and truststore respectively. You >> should end up with three keys in gateway.jks afterward; thy >> gateway-identity, nifi-key, and nifi-cert keys. Once both of those are >> added to gateway.jks, and you have configured the service definition for >> NiFi in your topology with useTwoWaySsl set to true, the two-way SSL >> handshake should succeed. >> >> Also, you will want to add the DN from that nifi-key as a node identity >> (in the same place you set the initial admin identity) so that NiFi can >> create a "user" to represent the Knox node and add a policy for you to >> allow that node/identity to proxy requests, if you haven't already done so. >> >> After adding the keystore and truststore material to gateway.jks, added a >> user and policy for NiFi to identify and authorize Knox for proxying, Knox >> should be able to proxy NiFi securely. >> >> On Thu, Mar 8, 2018 at 11:44 AM larry mccay wrote: >> >>> There was definitely some nuances to this area and you need to be >>> careful with confusing the truststore for Knox accepting client certs and >>> Knox sending client certs from dispatch to the back end services. You may >>> find better luck adding the nifi server cert or its ca cert to the Knox >>> machine cacerts truststore. >>> >>> I am explicitly adding @jeff here as well since he is the Nifi >>> integration guru. >>> >>> On Thu, Mar 8, 2018 at 11:09 AM, Ryan H < >>> ryan.howell.developm...@gmail.com> wrote: >>> Ok. Well I've imported the truststore from NiFi into the gateway keystore but I am still unable to connect to NiFi via Knox with the same error. Password for the NiFi truststore is the same as the gateway keystore just to make sure that wasn't an issue. I've changed some of the gateway-site.xml settings to explicitly trust certificates and to use the gateway.jks store, but still getting the same error. It's clearly an ssl error between Knox and NiFi. I'll just keep digging I suppose until it gets worked out. Not really sure what else to try here unfortunately... -Ryan On Thu, Mar 8, 2018 at 9:23 AM, Sandeep Moré wrote: > I am not that aware of NiFi specific SSL settings, but if you have a > truststore from NiFi (instead of a cert) you can use that, if it has a > deifferent password you will have to configure it. You can find the > instructions here > https://knox.apache.org/books/knox-1-0-0/user-guide.html#Mutual+Authentication+with+SSL > > > > On Wed, Mar 7, 2018 at 8:27 PM, Ryan H < > ryan.howell.developm...@gmail.com> wrote: > >> Hi Sandeep, >> >> So I have the NiFi TLS Toolkit running in Client/Server mode. I have >> made a request to the CA server from the Knox machine by running the TLS >> Toolkit as a Client and received a keystore, truststore, and >> nifi-cert.pem. >> I understand that I need to get the public cert into the Knox keystore, >> but >> unsure which one to import and to where. Should the cert be imported into >> the KNOX_HOME/data/security/keystores/gateway.jks store? And do you know >> which one
Re: KnoxSSO with NiFi error: PKIX path building failed...
Yes, it seems to be progress. The only problem is that the error has me confused, as the "Certificate for nifi-address" does in fact match the subject alternative names: [nifi-address]"! The address is in fact in the list of SANS. Why would the error say that it doesn't match when it does in fact match? For what it is worth, I am using actual IP's here as these are private EC2 internal instances. Any thought? javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't match any of the subject alternative names: [nifi-address] On Thu, Mar 8, 2018 at 1:47 PM, Sandeep Moré wrote: > This looks like a progress to me :) > This is a different exception from the previous (PKIX path building failed > ). > > I would let Jeff comment more on it as he is subject matter expert on > this, but just looking at the exception I think adding nifi-address and/or > hostname to SubjectAlternativeName(SAN) should fix it [1] > > [1] https://confluence.atlassian.com/jirakb/javax-net-ssl- > sslpeerunverifiedexception-certificate-for-hostname- > doesn-t-match-any-of-the-subject-alternative-names-939719154.html > > > > > > On Thu, Mar 8, 2018 at 12:51 PM, Ryan H > wrote: > >> Hi Jeff, >> >> Thanks for the additional info on this. Here is what I have done: >> >> 1. Imported the keystore.jks and truststore.jks obtained from the TLS >> Toolkit on the Knox machine into the Knox gateway.jks keystore >> 2. Added the DN used in step 1 above as "User Identity" and "Node >> Identity" in the authorizers.xml file on the NiFi machine >> 3. Set "enableTwoWaySsl" to true in sandbox.xml and NIFI service.xml >> 4. Restarted the Knox Gateway and the NiFi Cluster >> >> >> I am still getting the same error (I turned on DEBUG for gateway-log4j), >> but with some additional info (because I didn't have DEBUG on earlier, I am >> not sure if it is the same or different error): >> >> 2018-03-08 17:38:46,819 INFO service.knoxsso >> (WebSSOResource.java:addJWTHadoopCookie(304)) - JWT cookie successfully >> added. >> 2018-03-08 17:38:46,819 INFO service.knoxsso >> (WebSSOResource.java:getAuthenticationToken(214)) - About to redirect to >> original URL: https://knox-address:8443/gateway/sandbox/nifi-app/nifi >> 2018-03-08 17:38:46,902 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) >> - Received request: GET /nifi-app/nifi >> 2018-03-08 17:38:46,903 DEBUG federation.jwt >> (SSOCookieFederationFilter.java:getJWTFromCookie(139)) - hadoop-jwt >> Cookie has been found and is being processed. >> 2018-03-08 17:38:47,031 DEBUG knox.gateway >> (UrlRewriteProcessor.java:rewrite(163)) >> - Rewrote URL: https://knox-address:8443/gateway/sandbox/nifi-app/nifi, >> direction: IN via explicit rule: NIFI/nifi/inbound/path/query to URL: >> https://nifi-address:8443/nifi >> 2018-03-08 17:38:47,038 DEBUG knox.gateway >> (DefaultDispatch.java:executeOutboundRequest(121)) >> - Dispatch request: GET https://nifi-address:8443/nifi >> ?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72 >> 2018-03-08 17:38:47,180 WARN knox.gateway >> (DefaultDispatch.java:executeOutboundRequest(147)) >> - Connection exception dispatching request: >> https://nifi-address:8443/nifi?user.name=ba2d3b04-6bbd-4473- >> 80f4-c2f528cb1d72 javax.net.ssl.SSLPeerUnverifiedException: Certificate >> for doesn't match any of the subject alternative names: >> [nifi-address] >> javax.net.ssl.SSLPeerUnverifiedException: Certificate for >> doesn't match any of the subject alternative names: [nifi-address] >> at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHo >> stname(SSLConnectionSocketFactory.java:467) >> at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLa >> yeredSocket(SSLConnectionSocketFactory.java:397) >> at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectS >> ocket(SSLConnectionSocketFactory.java:355) >> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperato >> r.connect(DefaultHttpClientConnectionOperator.java:142) >> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager >> .connect(PoolingHttpClientConnectionManager.java:359) >> at org.apache.http.impl.execchain.MainClientExec.establishRoute >> (MainClientExec.java:381) >> at org.apache.http.impl.execchain.MainClientExec.execute( >> MainClientExec.java:237) >> at org.apache.http.impl.execchain.ProtocolExec.execute( >> ProtocolExec.java:185) >> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) >> at org.apache.http.impl.execchain.RedirectExec.execute( >> RedirectExec.java:111) >> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int >> ernalHttpClient.java:185) >> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >> eableHttpClient.java:83) >> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >> eableHttpClient.java:108) >> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >> eableHttpClient.java:56) >> at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutb >> oundRequest(DefaultDispatch.java:130) >> at or
Re: KnoxSSO with NiFi error: PKIX path building failed...
This looks like a progress to me :) This is a different exception from the previous (PKIX path building failed ). I would let Jeff comment more on it as he is subject matter expert on this, but just looking at the exception I think adding nifi-address and/or hostname to SubjectAlternativeName(SAN) should fix it [1] [1] https://confluence.atlassian.com/jirakb/javax-net-ssl-sslpeerunverifiedexception-certificate-for-hostname-doesn-t-match-any-of-the-subject-alternative-names-939719154.html On Thu, Mar 8, 2018 at 12:51 PM, Ryan H wrote: > Hi Jeff, > > Thanks for the additional info on this. Here is what I have done: > > 1. Imported the keystore.jks and truststore.jks obtained from the TLS > Toolkit on the Knox machine into the Knox gateway.jks keystore > 2. Added the DN used in step 1 above as "User Identity" and "Node > Identity" in the authorizers.xml file on the NiFi machine > 3. Set "enableTwoWaySsl" to true in sandbox.xml and NIFI service.xml > 4. Restarted the Knox Gateway and the NiFi Cluster > > > I am still getting the same error (I turned on DEBUG for gateway-log4j), > but with some additional info (because I didn't have DEBUG on earlier, I am > not sure if it is the same or different error): > > 2018-03-08 17:38:46,819 INFO service.knoxsso > (WebSSOResource.java:addJWTHadoopCookie(304)) > - JWT cookie successfully added. > 2018-03-08 17:38:46,819 INFO service.knoxsso > (WebSSOResource.java:getAuthenticationToken(214)) > - About to redirect to original URL: https://knox-address:8443/ > gateway/sandbox/nifi-app/nifi > 2018-03-08 17:38:46,902 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) > - Received request: GET /nifi-app/nifi > 2018-03-08 17:38:46,903 DEBUG federation.jwt > (SSOCookieFederationFilter.java:getJWTFromCookie(139)) > - hadoop-jwt Cookie has been found and is being processed. > 2018-03-08 17:38:47,031 DEBUG knox.gateway > (UrlRewriteProcessor.java:rewrite(163)) > - Rewrote URL: https://knox-address:8443/gateway/sandbox/nifi-app/nifi, > direction: IN via explicit rule: NIFI/nifi/inbound/path/query to URL: > https://nifi-address:8443/nifi > 2018-03-08 17:38:47,038 DEBUG knox.gateway > (DefaultDispatch.java:executeOutboundRequest(121)) > - Dispatch request: GET https://nifi-address:8443/ > nifi?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72 > 2018-03-08 17:38:47,180 WARN knox.gateway > (DefaultDispatch.java:executeOutboundRequest(147)) > - Connection exception dispatching request: https://nifi-address:8443/ > nifi?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72 > javax.net.ssl.SSLPeerUnverifiedException: > Certificate for doesn't match any of the subject alternative > names: [nifi-address] > javax.net.ssl.SSLPeerUnverifiedException: Certificate for > doesn't match any of the subject alternative names: [nifi-address] > at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname( > SSLConnectionSocketFactory.java:467) > at org.apache.http.conn.ssl.SSLConnectionSocketFactory. > createLayeredSocket(SSLConnectionSocketFactory.java:397) > at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket( > SSLConnectionSocketFactory.java:355) > at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect( > DefaultHttpClientConnectionOperator.java:142) > at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect( > PoolingHttpClientConnectionManager.java:359) > at org.apache.http.impl.execchain.MainClientExec. > establishRoute(MainClientExec.java:381) > at org.apache.http.impl.execchain.MainClientExec. > execute(MainClientExec.java:237) > at org.apache.http.impl.execchain.ProtocolExec. > execute(ProtocolExec.java:185) > at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) > at org.apache.http.impl.execchain.RedirectExec. > execute(RedirectExec.java:111) > at org.apache.http.impl.client.InternalHttpClient.doExecute( > InternalHttpClient.java:185) > at org.apache.http.impl.client.CloseableHttpClient.execute( > CloseableHttpClient.java:83) > at org.apache.http.impl.client.CloseableHttpClient.execute( > CloseableHttpClient.java:108) > at org.apache.http.impl.client.CloseableHttpClient.execute( > CloseableHttpClient.java:56) > at org.apache.knox.gateway.dispatch.DefaultDispatch. > executeOutboundRequest(DefaultDispatch.java:130) > at org.apache.knox.gateway.dispatch.NiFiDispatch. > executeRequest(NiFiDispatch.java:39) > at org.apache.knox.gateway.dispatch.DefaultDispatch. > doGet(DefaultDispatch.java:278) > at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$ > GetAdapter.doMethod(GatewayDispatchFilter.java:122) > at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter( > GatewayDispatchFilter.java:105) > at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter( > AbstractGatewayFilter.java:61) > at org.apache.knox.gateway.GatewayFilter$Holder.doFilter( > GatewayFilter.java:377) > at org.apache.knox.gateway.GatewayFilter$Chain.doFilter( > GatewayFilter.java:277) > at org.apache.knox.gateway.ide
Re: KnoxSSO with NiFi error: PKIX path building failed...
Hi Jeff, Thanks for the additional info on this. Here is what I have done: 1. Imported the keystore.jks and truststore.jks obtained from the TLS Toolkit on the Knox machine into the Knox gateway.jks keystore 2. Added the DN used in step 1 above as "User Identity" and "Node Identity" in the authorizers.xml file on the NiFi machine 3. Set "enableTwoWaySsl" to true in sandbox.xml and NIFI service.xml 4. Restarted the Knox Gateway and the NiFi Cluster I am still getting the same error (I turned on DEBUG for gateway-log4j), but with some additional info (because I didn't have DEBUG on earlier, I am not sure if it is the same or different error): 2018-03-08 17:38:46,819 INFO service.knoxsso (WebSSOResource.java:addJWTHadoopCookie(304)) - JWT cookie successfully added. 2018-03-08 17:38:46,819 INFO service.knoxsso (WebSSOResource.java:getAuthenticationToken(214)) - About to redirect to original URL: https://knox-address:8443/gateway/sandbox/nifi-app/nifi 2018-03-08 17:38:46,902 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) - Received request: GET /nifi-app/nifi 2018-03-08 17:38:46,903 DEBUG federation.jwt (SSOCookieFederationFilter.java:getJWTFromCookie(139)) - hadoop-jwt Cookie has been found and is being processed. 2018-03-08 17:38:47,031 DEBUG knox.gateway (UrlRewriteProcessor.java:rewrite(163)) - Rewrote URL: https://knox-address:8443/gateway/sandbox/nifi-app/nifi, direction: IN via explicit rule: NIFI/nifi/inbound/path/query to URL: https://nifi-address:8443/nifi 2018-03-08 17:38:47,038 DEBUG knox.gateway (DefaultDispatch.java:executeOutboundRequest(121)) - Dispatch request: GET https://nifi-address:8443/nifi?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72 2018-03-08 17:38:47,180 WARN knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) - Connection exception dispatching request: https://nifi-address:8443/nifi?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72 javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't match any of the subject alternative names: [nifi-address] javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't match any of the subject alternative names: [nifi-address] at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:467) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:397) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:130) at org.apache.knox.gateway.dispatch.NiFiDispatch.executeRequest(NiFiDispatch.java:39) at org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:278) at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:122) at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:105) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:277) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:196) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:153) at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:90) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:277) at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(Abstr
Re: KnoxSSO with NiFi error: PKIX path building failed...
Ryan, In addition to the things I mentioned in my previous message, there are some properties you'll need to set as well. In nifi.properties, set nifi.web.proxy.context.path to "/gateway/sandbox/nifi-app". The host and port of the Knox service should also be set for nifi.web.proxy.host. On Thu, Mar 8, 2018 at 12:06 PM Jeff wrote: > Ryan, > > I noticed in your sandbox.xml, you have the "useTwoWaySsl" parameter set > to false. That needs to be set to true so that the dispatch will create an > SSL context that uses the keystore and truststore material in gateway.jks. > > I'm glad you're using the TLS Toolkit, I was going to suggest you give > that a try, initially. The cert from the keystore generated by the toolkit > that identifies the cert to use for Knox needs to be added to gateway.jks, > along with the nifi-cert key from the truststore. Just importing both the > keystore and truststore generated by the toolkit for Knox should be all you > have to do there, since the toolkit generates those stores with just the > nifi-key and nifi-cert in the keystore and truststore respectively. You > should end up with three keys in gateway.jks afterward; thy > gateway-identity, nifi-key, and nifi-cert keys. Once both of those are > added to gateway.jks, and you have configured the service definition for > NiFi in your topology with useTwoWaySsl set to true, the two-way SSL > handshake should succeed. > > Also, you will want to add the DN from that nifi-key as a node identity > (in the same place you set the initial admin identity) so that NiFi can > create a "user" to represent the Knox node and add a policy for you to > allow that node/identity to proxy requests, if you haven't already done so. > > After adding the keystore and truststore material to gateway.jks, added a > user and policy for NiFi to identify and authorize Knox for proxying, Knox > should be able to proxy NiFi securely. > > On Thu, Mar 8, 2018 at 11:44 AM larry mccay wrote: > >> There was definitely some nuances to this area and you need to be careful >> with confusing the truststore for Knox accepting client certs and Knox >> sending client certs from dispatch to the back end services. You may find >> better luck adding the nifi server cert or its ca cert to the Knox machine >> cacerts truststore. >> >> I am explicitly adding @jeff here as well since he is the Nifi >> integration guru. >> >> On Thu, Mar 8, 2018 at 11:09 AM, Ryan H < >> ryan.howell.developm...@gmail.com> wrote: >> >>> Ok. Well I've imported the truststore from NiFi into the gateway >>> keystore but I am still unable to connect to NiFi via Knox with the same >>> error. Password for the NiFi truststore is the same as the gateway keystore >>> just to make sure that wasn't an issue. I've changed some of the >>> gateway-site.xml settings to explicitly trust certificates and to use the >>> gateway.jks store, but still getting the same error. It's clearly an ssl >>> error between Knox and NiFi. I'll just keep digging I suppose until it gets >>> worked out. Not really sure what else to try here unfortunately... >>> >>> -Ryan >>> >>> On Thu, Mar 8, 2018 at 9:23 AM, Sandeep Moré >>> wrote: >>> I am not that aware of NiFi specific SSL settings, but if you have a truststore from NiFi (instead of a cert) you can use that, if it has a deifferent password you will have to configure it. You can find the instructions here https://knox.apache.org/books/knox-1-0-0/user-guide.html#Mutual+Authentication+with+SSL On Wed, Mar 7, 2018 at 8:27 PM, Ryan H < ryan.howell.developm...@gmail.com> wrote: > Hi Sandeep, > > So I have the NiFi TLS Toolkit running in Client/Server mode. I have > made a request to the CA server from the Knox machine by running the TLS > Toolkit as a Client and received a keystore, truststore, and > nifi-cert.pem. > I understand that I need to get the public cert into the Knox keystore, > but > unsure which one to import and to where. Should the cert be imported into > the KNOX_HOME/data/security/keystores/gateway.jks store? And do you know > which one of the files should have the public cert? > > Thanks in Advance, > > -Ryan > > On Wed, Mar 7, 2018 at 8:03 PM, Sandeep Moré > wrote: > >> Hello Ryan, >> >> Looks like you need to provision NiFi public cert into Knox keystore >> that should do it. >> >> >> On Wed, Mar 7, 2018 at 7:12 PM, Ryan H < >> ryan.howell.developm...@gmail.com> wrote: >> >>> Hi All, >>> >>> I seem to be having a really tough time getting Knox to work with a >>> secure NiFi cluster set up. I have tried to get this working two >>> different >>> ways. Both ways have basically the same set up for knoxsso, where it >>> uses >>> cloud foundry UAA as an external identity provider (currently configured >>> for OpenID, with the /.well-known/openid-configuratio
Re: KnoxSSO with NiFi error: PKIX path building failed...
Ryan, I noticed in your sandbox.xml, you have the "useTwoWaySsl" parameter set to false. That needs to be set to true so that the dispatch will create an SSL context that uses the keystore and truststore material in gateway.jks. I'm glad you're using the TLS Toolkit, I was going to suggest you give that a try, initially. The cert from the keystore generated by the toolkit that identifies the cert to use for Knox needs to be added to gateway.jks, along with the nifi-cert key from the truststore. Just importing both the keystore and truststore generated by the toolkit for Knox should be all you have to do there, since the toolkit generates those stores with just the nifi-key and nifi-cert in the keystore and truststore respectively. You should end up with three keys in gateway.jks afterward; thy gateway-identity, nifi-key, and nifi-cert keys. Once both of those are added to gateway.jks, and you have configured the service definition for NiFi in your topology with useTwoWaySsl set to true, the two-way SSL handshake should succeed. Also, you will want to add the DN from that nifi-key as a node identity (in the same place you set the initial admin identity) so that NiFi can create a "user" to represent the Knox node and add a policy for you to allow that node/identity to proxy requests, if you haven't already done so. After adding the keystore and truststore material to gateway.jks, added a user and policy for NiFi to identify and authorize Knox for proxying, Knox should be able to proxy NiFi securely. On Thu, Mar 8, 2018 at 11:44 AM larry mccay wrote: > There was definitely some nuances to this area and you need to be careful > with confusing the truststore for Knox accepting client certs and Knox > sending client certs from dispatch to the back end services. You may find > better luck adding the nifi server cert or its ca cert to the Knox machine > cacerts truststore. > > I am explicitly adding @jeff here as well since he is the Nifi integration > guru. > > On Thu, Mar 8, 2018 at 11:09 AM, Ryan H > wrote: > >> Ok. Well I've imported the truststore from NiFi into the gateway keystore >> but I am still unable to connect to NiFi via Knox with the same error. >> Password for the NiFi truststore is the same as the gateway keystore just >> to make sure that wasn't an issue. I've changed some of the >> gateway-site.xml settings to explicitly trust certificates and to use the >> gateway.jks store, but still getting the same error. It's clearly an ssl >> error between Knox and NiFi. I'll just keep digging I suppose until it gets >> worked out. Not really sure what else to try here unfortunately... >> >> -Ryan >> >> On Thu, Mar 8, 2018 at 9:23 AM, Sandeep Moré >> wrote: >> >>> I am not that aware of NiFi specific SSL settings, but if you have a >>> truststore from NiFi (instead of a cert) you can use that, if it has a >>> deifferent password you will have to configure it. You can find the >>> instructions here >>> https://knox.apache.org/books/knox-1-0-0/user-guide.html#Mutual+Authentication+with+SSL >>> >>> >>> >>> On Wed, Mar 7, 2018 at 8:27 PM, Ryan H < >>> ryan.howell.developm...@gmail.com> wrote: >>> Hi Sandeep, So I have the NiFi TLS Toolkit running in Client/Server mode. I have made a request to the CA server from the Knox machine by running the TLS Toolkit as a Client and received a keystore, truststore, and nifi-cert.pem. I understand that I need to get the public cert into the Knox keystore, but unsure which one to import and to where. Should the cert be imported into the KNOX_HOME/data/security/keystores/gateway.jks store? And do you know which one of the files should have the public cert? Thanks in Advance, -Ryan On Wed, Mar 7, 2018 at 8:03 PM, Sandeep Moré wrote: > Hello Ryan, > > Looks like you need to provision NiFi public cert into Knox keystore > that should do it. > > > On Wed, Mar 7, 2018 at 7:12 PM, Ryan H < > ryan.howell.developm...@gmail.com> wrote: > >> Hi All, >> >> I seem to be having a really tough time getting Knox to work with a >> secure NiFi cluster set up. I have tried to get this working two >> different >> ways. Both ways have basically the same set up for knoxsso, where it uses >> cloud foundry UAA as an external identity provider (currently configured >> for OpenID, with the /.well-known/openid-configuration prepended to the >> UAA >> instance url). I'm not sure if OpenID connect is the correct way to go, I >> believe there are other options with UAA; this is just the route I went >> as >> I initially was going to configure NiFi OpenID properties with my UAA >> instance. I have since decided (based on other factors) that Knox would >> be >> a better way to go. I have been focusing on option 1 below, as I think >> this >> is the preferred way. However, I tried option 2 below j
Re: KnoxSSO with NiFi error: PKIX path building failed...
There was definitely some nuances to this area and you need to be careful with confusing the truststore for Knox accepting client certs and Knox sending client certs from dispatch to the back end services. You may find better luck adding the nifi server cert or its ca cert to the Knox machine cacerts truststore. I am explicitly adding @jeff here as well since he is the Nifi integration guru. On Thu, Mar 8, 2018 at 11:09 AM, Ryan H wrote: > Ok. Well I've imported the truststore from NiFi into the gateway keystore > but I am still unable to connect to NiFi via Knox with the same error. > Password for the NiFi truststore is the same as the gateway keystore just > to make sure that wasn't an issue. I've changed some of the > gateway-site.xml settings to explicitly trust certificates and to use the > gateway.jks store, but still getting the same error. It's clearly an ssl > error between Knox and NiFi. I'll just keep digging I suppose until it gets > worked out. Not really sure what else to try here unfortunately... > > -Ryan > > On Thu, Mar 8, 2018 at 9:23 AM, Sandeep Moré > wrote: > >> I am not that aware of NiFi specific SSL settings, but if you have a >> truststore from NiFi (instead of a cert) you can use that, if it has a >> deifferent password you will have to configure it. You can find the >> instructions here https://knox.apache.org/books/knox-1-0-0/user-guide.htm >> l#Mutual+Authentication+with+SSL >> >> >> >> On Wed, Mar 7, 2018 at 8:27 PM, Ryan H > > wrote: >> >>> Hi Sandeep, >>> >>> So I have the NiFi TLS Toolkit running in Client/Server mode. I have >>> made a request to the CA server from the Knox machine by running the TLS >>> Toolkit as a Client and received a keystore, truststore, and nifi-cert.pem. >>> I understand that I need to get the public cert into the Knox keystore, but >>> unsure which one to import and to where. Should the cert be imported into >>> the KNOX_HOME/data/security/keystores/gateway.jks store? And do you >>> know which one of the files should have the public cert? >>> >>> Thanks in Advance, >>> >>> -Ryan >>> >>> On Wed, Mar 7, 2018 at 8:03 PM, Sandeep Moré >>> wrote: >>> Hello Ryan, Looks like you need to provision NiFi public cert into Knox keystore that should do it. On Wed, Mar 7, 2018 at 7:12 PM, Ryan H >>> .com> wrote: > Hi All, > > I seem to be having a really tough time getting Knox to work with a > secure NiFi cluster set up. I have tried to get this working two different > ways. Both ways have basically the same set up for knoxsso, where it uses > cloud foundry UAA as an external identity provider (currently configured > for OpenID, with the /.well-known/openid-configuration prepended to > the UAA instance url). I'm not sure if OpenID connect is the correct way > to > go, I believe there are other options with UAA; this is just the route I > went as I initially was going to configure NiFi OpenID properties with my > UAA instance. I have since decided (based on other factors) that Knox > would > be a better way to go. I have been focusing on option 1 below, as I think > this is the preferred way. However, I tried option 2 below just to see if > I > could get around the error temporarily. I've included the errors I am > running into below as well as relevant config. Any help is greatly > appreciated. > > versions: NiFi 1.6 and Knox 1.1.0 > > *1. Users will always access NiFi thru Knox (preferred)* > *Issue Facing: Getting "PKIX path building failed: unable to find > valid certification path to requested target"* > > *knoxsso.xml* > > > > webappsec > WebAppSec > true > xframe.options.enabledtrue > > > > federation > pac4j > true > > pac4j.session.store > J2ESessionStore > > > pac4j.callbackUrl > https://my-knox-host:8443/gateway/knoxsso/api/v1/webs > so > > > clientName > OidcClient > > > oidc.id > some_client_id > > > oidc.secret > some_client_secret > > > oidc.discoveryUri > https://my-uaa-host:443/.well-known/openid-configurat > ion > > > oidc.preferredJwsAlgorithm > RS256 > > > > > > knoxauth > > > KNOXSSO > > knoxsso.cookie.secure.only > false > > > knoxsso.enable.session > true > > > knoxsso.cookie.max.age > session > > >
Re: KnoxSSO with NiFi error: PKIX path building failed...
Ok. Well I've imported the truststore from NiFi into the gateway keystore
but I am still unable to connect to NiFi via Knox with the same error.
Password for the NiFi truststore is the same as the gateway keystore just
to make sure that wasn't an issue. I've changed some of the
gateway-site.xml settings to explicitly trust certificates and to use the
gateway.jks store, but still getting the same error. It's clearly an ssl
error between Knox and NiFi. I'll just keep digging I suppose until it gets
worked out. Not really sure what else to try here unfortunately...
-Ryan
On Thu, Mar 8, 2018 at 9:23 AM, Sandeep Moré wrote:
> I am not that aware of NiFi specific SSL settings, but if you have a
> truststore from NiFi (instead of a cert) you can use that, if it has a
> deifferent password you will have to configure it. You can find the
> instructions here https://knox.apache.org/books/knox-1-0-0/user-guide.
> html#Mutual+Authentication+with+SSL
>
>
>
> On Wed, Mar 7, 2018 at 8:27 PM, Ryan H
> wrote:
>
>> Hi Sandeep,
>>
>> So I have the NiFi TLS Toolkit running in Client/Server mode. I have made
>> a request to the CA server from the Knox machine by running the TLS Toolkit
>> as a Client and received a keystore, truststore, and nifi-cert.pem. I
>> understand that I need to get the public cert into the Knox keystore, but
>> unsure which one to import and to where. Should the cert be imported into
>> the KNOX_HOME/data/security/keystores/gateway.jks store? And do you know
>> which one of the files should have the public cert?
>>
>> Thanks in Advance,
>>
>> -Ryan
>>
>> On Wed, Mar 7, 2018 at 8:03 PM, Sandeep Moré
>> wrote:
>>
>>> Hello Ryan,
>>>
>>> Looks like you need to provision NiFi public cert into Knox keystore
>>> that should do it.
>>>
>>>
>>> On Wed, Mar 7, 2018 at 7:12 PM, Ryan H >> .com> wrote:
>>>
Hi All,
I seem to be having a really tough time getting Knox to work with a
secure NiFi cluster set up. I have tried to get this working two different
ways. Both ways have basically the same set up for knoxsso, where it uses
cloud foundry UAA as an external identity provider (currently configured
for OpenID, with the /.well-known/openid-configuration prepended to
the UAA instance url). I'm not sure if OpenID connect is the correct way to
go, I believe there are other options with UAA; this is just the route I
went as I initially was going to configure NiFi OpenID properties with my
UAA instance. I have since decided (based on other factors) that Knox would
be a better way to go. I have been focusing on option 1 below, as I think
this is the preferred way. However, I tried option 2 below just to see if I
could get around the error temporarily. I've included the errors I am
running into below as well as relevant config. Any help is greatly
appreciated.
versions: NiFi 1.6 and Knox 1.1.0
*1. Users will always access NiFi thru Knox (preferred)*
*Issue Facing: Getting "PKIX path building failed: unable to find valid
certification path to requested target"*
*knoxsso.xml*
webappsec
WebAppSec
true
xframe.options.enabledtrue>>> >
federation
pac4j
true
pac4j.session.store
J2ESessionStore
pac4j.callbackUrl
https://my-knox-host:8443/gateway/knoxsso/api/v1/webs
so
clientName
OidcClient
oidc.id
some_client_id
oidc.secret
some_client_secret
oidc.discoveryUri
https://my-uaa-host:443/.well-known/openid-configurat
ion
oidc.preferredJwsAlgorithm
RS256
knoxauth
KNOXSSO
knoxsso.cookie.secure.only
false
knoxsso.enable.session
true
knoxsso.cookie.max.age
session
knoxsso.token.ttl
360
knoxsso.redirect.whitelist.regex
^https?:\/\/(localhost|10\.227\.85\.2|[0-9]{1,3}\.[0
-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}||127\.0\.0\.1|0:0:0:0:0:0:0
:1|::1):[0-9].*$
*sandbox.xml*
federation
SSOCookieProvider
true
sso.authentication.provider.url
https://my-knox-host:8443/gateway/knoxsso/api/v1/webs
so
identity-assertion
Re: KnoxSSO with NiFi error: PKIX path building failed...
I am not that aware of NiFi specific SSL settings, but if you have a
truststore from NiFi (instead of a cert) you can use that, if it has a
deifferent password you will have to configure it. You can find the
instructions here
https://knox.apache.org/books/knox-1-0-0/user-guide.html#Mutual+Authentication+with+SSL
On Wed, Mar 7, 2018 at 8:27 PM, Ryan H
wrote:
> Hi Sandeep,
>
> So I have the NiFi TLS Toolkit running in Client/Server mode. I have made
> a request to the CA server from the Knox machine by running the TLS Toolkit
> as a Client and received a keystore, truststore, and nifi-cert.pem. I
> understand that I need to get the public cert into the Knox keystore, but
> unsure which one to import and to where. Should the cert be imported into
> the KNOX_HOME/data/security/keystores/gateway.jks store? And do you know
> which one of the files should have the public cert?
>
> Thanks in Advance,
>
> -Ryan
>
> On Wed, Mar 7, 2018 at 8:03 PM, Sandeep Moré
> wrote:
>
>> Hello Ryan,
>>
>> Looks like you need to provision NiFi public cert into Knox keystore that
>> should do it.
>>
>>
>> On Wed, Mar 7, 2018 at 7:12 PM, Ryan H > > wrote:
>>
>>> Hi All,
>>>
>>> I seem to be having a really tough time getting Knox to work with a
>>> secure NiFi cluster set up. I have tried to get this working two different
>>> ways. Both ways have basically the same set up for knoxsso, where it uses
>>> cloud foundry UAA as an external identity provider (currently configured
>>> for OpenID, with the /.well-known/openid-configuration prepended to the
>>> UAA instance url). I'm not sure if OpenID connect is the correct way to go,
>>> I believe there are other options with UAA; this is just the route I went
>>> as I initially was going to configure NiFi OpenID properties with my UAA
>>> instance. I have since decided (based on other factors) that Knox would be
>>> a better way to go. I have been focusing on option 1 below, as I think this
>>> is the preferred way. However, I tried option 2 below just to see if I
>>> could get around the error temporarily. I've included the errors I am
>>> running into below as well as relevant config. Any help is greatly
>>> appreciated.
>>>
>>> versions: NiFi 1.6 and Knox 1.1.0
>>>
>>> *1. Users will always access NiFi thru Knox (preferred)*
>>> *Issue Facing: Getting "PKIX path building failed: unable to find valid
>>> certification path to requested target"*
>>>
>>> *knoxsso.xml*
>>>
>>>
>>>
>>> webappsec
>>> WebAppSec
>>> true
>>> xframe.options.enabledtrue>> >
>>>
>>>
>>> federation
>>> pac4j
>>> true
>>>
>>> pac4j.session.store
>>> J2ESessionStore
>>>
>>>
>>> pac4j.callbackUrl
>>> https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
>>>
>>>
>>>
>>> clientName
>>> OidcClient
>>>
>>>
>>> oidc.id
>>> some_client_id
>>>
>>>
>>> oidc.secret
>>> some_client_secret
>>>
>>>
>>> oidc.discoveryUri
>>> https://my-uaa-host:443/.well-known/openid-configurat
>>> ion
>>>
>>>
>>> oidc.preferredJwsAlgorithm
>>> RS256
>>>
>>>
>>>
>>>
>>>
>>> knoxauth
>>>
>>>
>>> KNOXSSO
>>>
>>> knoxsso.cookie.secure.only
>>> false
>>>
>>>
>>> knoxsso.enable.session
>>> true
>>>
>>>
>>> knoxsso.cookie.max.age
>>> session
>>>
>>>
>>> knoxsso.token.ttl
>>> 360
>>>
>>>
>>>knoxsso.redirect.whitelist.regex
>>>^https?:\/\/(localhost|10\.227\.85\.2|[0-9]{1,3}\.[0
>>> -9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}||127\.0\.0\.1|0:0:0:0:0:0:
>>> 0:1|::1):[0-9].*$
>>>
>>>
>>>
>>>
>>> *sandbox.xml*
>>>
>>> federation
>>> SSOCookieProvider
>>> true
>>>
>>> sso.authentication.provider.url
>>> https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
>>>
>>>
>>>
>>>
>>>
>>>
>>> identity-assertion
>>> Default
>>> true
>>>
>>>
>>>
>>> hostmap
>>> static
>>> true
>>>
>>>
>>>
>>>
>>>
>>> NIFI
>>> https://my-nifi-host:8443
>>>
>>>
>>>
>>> *Stacktrace from Knox:*
>>> knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) -
>>> Connection exception dispatching request: https://my-nifi-host:8443/nifi
>>> ?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72
>>> javax.net.ssl.SSLHandshakeException:
>>> sun.security.validator.ValidatorException:
>>> PKIX path building failed:
>>> sun.security.provider.certpath.SunCertPathBuilderException:
>>> unable to find valid certification path to requested target
>>> javax.net.ssl.SSLHandshakeException:
>>> sun.security.validator
Re: KnoxSSO with NiFi error: PKIX path building failed...
Based on the error I am getting from Knox, does it lead to be that it is surely a certificate error? I've imported both the truststore.jks and keystore.jks into the gateway.jks (used a config file so as to have the same password as the Knox master password), restarted the Gateway, and still getting the same error. 2018-03-08 03:47:14,284 WARN knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) - Connection exception dispatching request: https://my-nifi-host:8443/nifi?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72 javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:328) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1614) at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052) at sun.security.ssl.Handshaker.process_record(Handshaker.java:987) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:130) at org.apache.knox.gateway.dispatch.NiFiDispatch.executeRequest(NiFiDispatch.java:39) at org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:278) at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:122) at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:105) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:277) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:196) at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:153) at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:90) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:277) at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:277) at org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter$1.run(AbstractJWTFilter.java:202) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.knox.gateway.prov
Re: KnoxSSO with NiFi error: PKIX path building failed...
Hi Sandeep,
So I have the NiFi TLS Toolkit running in Client/Server mode. I have made a
request to the CA server from the Knox machine by running the TLS Toolkit
as a Client and received a keystore, truststore, and nifi-cert.pem. I
understand that I need to get the public cert into the Knox keystore, but
unsure which one to import and to where. Should the cert be imported into
the KNOX_HOME/data/security/keystores/gateway.jks store? And do you know
which one of the files should have the public cert?
Thanks in Advance,
-Ryan
On Wed, Mar 7, 2018 at 8:03 PM, Sandeep Moré wrote:
> Hello Ryan,
>
> Looks like you need to provision NiFi public cert into Knox keystore that
> should do it.
>
>
> On Wed, Mar 7, 2018 at 7:12 PM, Ryan H
> wrote:
>
>> Hi All,
>>
>> I seem to be having a really tough time getting Knox to work with a
>> secure NiFi cluster set up. I have tried to get this working two different
>> ways. Both ways have basically the same set up for knoxsso, where it uses
>> cloud foundry UAA as an external identity provider (currently configured
>> for OpenID, with the /.well-known/openid-configuration prepended to the
>> UAA instance url). I'm not sure if OpenID connect is the correct way to go,
>> I believe there are other options with UAA; this is just the route I went
>> as I initially was going to configure NiFi OpenID properties with my UAA
>> instance. I have since decided (based on other factors) that Knox would be
>> a better way to go. I have been focusing on option 1 below, as I think this
>> is the preferred way. However, I tried option 2 below just to see if I
>> could get around the error temporarily. I've included the errors I am
>> running into below as well as relevant config. Any help is greatly
>> appreciated.
>>
>> versions: NiFi 1.6 and Knox 1.1.0
>>
>> *1. Users will always access NiFi thru Knox (preferred)*
>> *Issue Facing: Getting "PKIX path building failed: unable to find valid
>> certification path to requested target"*
>>
>> *knoxsso.xml*
>>
>>
>>
>> webappsec
>> WebAppSec
>> true
>> xframe.options.enabledtrue> >
>>
>>
>> federation
>> pac4j
>> true
>>
>> pac4j.session.store
>> J2ESessionStore
>>
>>
>> pac4j.callbackUrl
>> https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
>>
>>
>>
>> clientName
>> OidcClient
>>
>>
>> oidc.id
>> some_client_id
>>
>>
>> oidc.secret
>> some_client_secret
>>
>>
>> oidc.discoveryUri
>> https://my-uaa-host:443/.well-known/openid-configuration
>>
>>
>>
>> oidc.preferredJwsAlgorithm
>> RS256
>>
>>
>>
>>
>>
>> knoxauth
>>
>>
>> KNOXSSO
>>
>> knoxsso.cookie.secure.only
>> false
>>
>>
>> knoxsso.enable.session
>> true
>>
>>
>> knoxsso.cookie.max.age
>> session
>>
>>
>> knoxsso.token.ttl
>> 360
>>
>>
>>knoxsso.redirect.whitelist.regex
>>^https?:\/\/(localhost|10\.227\.85\.2|[0-9]{1,3}\.[
>> 0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}||127\.0\.0\.1|0:0:0:0:0:
>> 0:0:1|::1):[0-9].*$
>>
>>
>>
>>
>> *sandbox.xml*
>>
>> federation
>> SSOCookieProvider
>> true
>>
>> sso.authentication.provider.url
>> https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
>>
>>
>>
>>
>>
>>
>> identity-assertion
>> Default
>> true
>>
>>
>>
>> hostmap
>> static
>> true
>>
>>
>>
>>
>>
>> NIFI
>> https://my-nifi-host:8443
>>
>>
>>
>> *Stacktrace from Knox:*
>> knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) -
>> Connection exception dispatching request: https://my-nifi-host:8443/nifi
>> ?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72
>> javax.net.ssl.SSLHandshakeException:
>> sun.security.validator.ValidatorException:
>> PKIX path building failed:
>> sun.security.provider.certpath.SunCertPathBuilderException:
>> unable to find valid certification path to requested target
>> javax.net.ssl.SSLHandshakeException:
>> sun.security.validator.ValidatorException:
>> PKIX path building failed:
>> sun.security.provider.certpath.SunCertPathBuilderException:
>> unable to find valid certification path to requested target
>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
>> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:328)
>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322)
>> at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHa
>> ndshaker.java:1614)
>> at sun.security.ssl
Re: KnoxSSO with NiFi error: PKIX path building failed...
Hello Ryan,
Looks like you need to provision NiFi public cert into Knox keystore that
should do it.
On Wed, Mar 7, 2018 at 7:12 PM, Ryan H
wrote:
> Hi All,
>
> I seem to be having a really tough time getting Knox to work with a secure
> NiFi cluster set up. I have tried to get this working two different ways.
> Both ways have basically the same set up for knoxsso, where it uses cloud
> foundry UAA as an external identity provider (currently configured for
> OpenID, with the /.well-known/openid-configuration prepended to the UAA
> instance url). I'm not sure if OpenID connect is the correct way to go, I
> believe there are other options with UAA; this is just the route I went as
> I initially was going to configure NiFi OpenID properties with my UAA
> instance. I have since decided (based on other factors) that Knox would be
> a better way to go. I have been focusing on option 1 below, as I think this
> is the preferred way. However, I tried option 2 below just to see if I
> could get around the error temporarily. I've included the errors I am
> running into below as well as relevant config. Any help is greatly
> appreciated.
>
> versions: NiFi 1.6 and Knox 1.1.0
>
> *1. Users will always access NiFi thru Knox (preferred)*
> *Issue Facing: Getting "PKIX path building failed: unable to find valid
> certification path to requested target"*
>
> *knoxsso.xml*
>
>
>
> webappsec
> WebAppSec
> true
> xframe.options.enabledtrue value>
>
>
> federation
> pac4j
> true
>
> pac4j.session.store
> J2ESessionStore
>
>
> pac4j.callbackUrl
> https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
>
>
>
> clientName
> OidcClient
>
>
> oidc.id
> some_client_id
>
>
> oidc.secret
> some_client_secret
>
>
> oidc.discoveryUri
> https://my-uaa-host:443/.well-known/openid-configuration
>
>
>
> oidc.preferredJwsAlgorithm
> RS256
>
>
>
>
>
> knoxauth
>
>
> KNOXSSO
>
> knoxsso.cookie.secure.only
> false
>
>
> knoxsso.enable.session
> true
>
>
> knoxsso.cookie.max.age
> session
>
>
> knoxsso.token.ttl
> 360
>
>
>knoxsso.redirect.whitelist.regex
>^https?:\/\/(localhost|10\.227\.85\.2|[0-9]
> {1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}||127\.0\.0\.1|0:0:
> 0:0:0:0:0:1|::1):[0-9].*$
>
>
>
>
> *sandbox.xml*
>
> federation
> SSOCookieProvider
> true
>
> sso.authentication.provider.url
> https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
>
>
>
>
>
>
> identity-assertion
> Default
> true
>
>
>
> hostmap
> static
> true
>
>
>
>
>
> NIFI
> https://my-nifi-host:8443
>
>
>
> *Stacktrace from Knox:*
> knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) -
> Connection exception dispatching request: https://my-nifi-host:8443/
> nifi?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72
> javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to
> find valid certification path to requested target
> javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException:
> unable to find valid certification path to requested target
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:328)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322)
> at sun.security.ssl.ClientHandshaker.serverCertificate(
> ClientHandshaker.java:1614)
> at sun.security.ssl.ClientHandshaker.processMessage(
> ClientHandshaker.java:216)
> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
> at sun.security.ssl.Handshaker.process_record(Handshaker.java:987)
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
> at sun.security.ssl.SSLSocketImpl.performInitialHandshake(
> SSLSocketImpl.java:1385)
> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
> at org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> createLayeredSocket(SSLConnectionSocketFactory.java:396)
> at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(
> SSLConnectionSocketFactory.java:355)
> at org.apache.
KnoxSSO with NiFi error: PKIX path building failed...
Hi All,
I seem to be having a really tough time getting Knox to work with a secure
NiFi cluster set up. I have tried to get this working two different ways.
Both ways have basically the same set up for knoxsso, where it uses cloud
foundry UAA as an external identity provider (currently configured for
OpenID, with the /.well-known/openid-configuration prepended to the UAA
instance url). I'm not sure if OpenID connect is the correct way to go, I
believe there are other options with UAA; this is just the route I went as
I initially was going to configure NiFi OpenID properties with my UAA
instance. I have since decided (based on other factors) that Knox would be
a better way to go. I have been focusing on option 1 below, as I think this
is the preferred way. However, I tried option 2 below just to see if I
could get around the error temporarily. I've included the errors I am
running into below as well as relevant config. Any help is greatly
appreciated.
versions: NiFi 1.6 and Knox 1.1.0
*1. Users will always access NiFi thru Knox (preferred)*
*Issue Facing: Getting "PKIX path building failed: unable to find valid
certification path to requested target"*
*knoxsso.xml*
webappsec
WebAppSec
true
xframe.options.enabledtrue
federation
pac4j
true
pac4j.session.store
J2ESessionStore
pac4j.callbackUrl
https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
clientName
OidcClient
oidc.id
some_client_id
oidc.secret
some_client_secret
oidc.discoveryUri
https://my-uaa-host:443/.well-known/openid-configuration
oidc.preferredJwsAlgorithm
RS256
knoxauth
KNOXSSO
knoxsso.cookie.secure.only
false
knoxsso.enable.session
true
knoxsso.cookie.max.age
session
knoxsso.token.ttl
360
knoxsso.redirect.whitelist.regex
^https?:\/\/(localhost|10\.227\.85\.2|[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}||127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$
*sandbox.xml*
federation
SSOCookieProvider
true
sso.authentication.provider.url
https://my-knox-host:8443/gateway/knoxsso/api/v1/websso
identity-assertion
Default
true
hostmap
static
true
NIFI
https://my-nifi-host:8443
*Stacktrace from Knox:*
knox.gateway (DefaultDispatch.java:executeOutboundRequest(147)) -
Connection exception dispatching request:
https://my-nifi-host:8443/nifi?user.name=ba2d3b04-6bbd-4473-80f4-c2f528cb1d72
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:328)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322)
at
sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1614)
at
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:987)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
at
org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
at
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
at
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.ex
