Just a quick update. I have added the below property in knoxsso.xml – suggested
in mail archives
<param>
<name>pac4j.session.store</name>
<value>J2ESessionStore</value>
</param>
After this config, it’s getting redirected to the expected URL after
authentication. But I’m facing other issue related to regex whitelist.
Attached the log below.
****************
<< gateway.log >>
***************
2018-07-18 04:29:39,917 DEBUG filter.Pac4jIdentityAdapter
(Pac4jIdentityAdapter.java:doFilter(85)) - User authenticated as:
#SAML2Profile# | id: [email protected] | attributes:
{Mail=[[email protected]], notOnOrAfter=2018-07-18T04:34:39.599Z,
sessionindex=mmocFXXZSWoAxo4el4O1KdbThm-, notBefore=2018-07-18T04:24:39.599Z} |
roles: [] | permissions: [] | isRemembered: false | clientName: SAML2Client |
linkedId: null |
2018-07-18 04:29:40,729 INFO service.knoxsso
(WebSSOResource.java:getCookieValue(330)) - Unable to find cookie with name:
original-url
2018-07-18 04:29:40,730 ERROR service.knoxsso
(WebSSOResource.java:getAuthenticationToken(188)) - The original URL:
https://<dnsname>:8446/gateway/gate1/yarn/ for redirecting back after
authentication is not valid according to the configured whitelist:
^https?:\/\/(localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$. See
documentation for KnoxSSO Whitelisting.
Thanks,
Praveen.R
From: "Ravikumar, Praveen Krishnamoorthy" <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Tuesday, July 17, 2018 at 7:47 PM
To: "[email protected]" <[email protected]>
Cc: "[email protected]" <[email protected]>, "Mohanan, Mahesh"
<[email protected]>
Subject: Re: Need help in enabling SAML auth in Apache Knox
Larry,
Thanks a lot for your swift response. As you listed below, first 4 steps
happening in order when I try to access the YARNUI through knox but I’m seeing
the below error in logs after the SAML assertion is posted.
2018-07-18 01:41:08,978 ERROR engine.DefaultCallbackLogic
(DefaultCallbackLogic.java:renewSession(123)) - Unable to renew the session.
The session store may not support this feature
Will this be a reason for this issue. Could you please provide your thoughts?
I have posted the log messages below for better understanding. Please let me
know, If you need any more inputs from my end.
****************
<< gateway.log >>
***************
2018-07-18 01:41:06,046 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) -
Received request: GET /yarn/
2018-07-18 01:41:06,047 DEBUG federation.jwt
(SSOCookieFederationFilter.java:doFilter(114)) - Sending redirect to:
https://<dnsName>:8446/gateway/knoxsso/api/v1/websso?originalUrl=https://<dnsName>:8446/gateway/gate1/yarn/
2018-07-18 01:41:06,111 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) -
Received request: GET /api/v1/websso
2018-07-18 01:41:06,111 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(101)) - Get from session: pac4jUserProfiles = null
2018-07-18 01:41:06,111 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(130)) - Save in session: pac4jRequestedUrl =
https://<dnsName>:8446/gateway/knoxsso/api/v1/websso?originalUrl=https://<dnsName>:8446/gateway/gate1/yarn/
2018-07-18 01:41:06,345 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(101)) - Get from session:
SAML2Client$attemptedAuthentication = null
2018-07-18 01:41:06,346 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(101)) - Get from session: samlRelayState = null
2018-07-18 01:41:06,346 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(130)) - Save in session: samlRelayState =
2018-07-18 01:41:08,679 DEBUG knox.gateway (GatewayFilter.java:doFilter(119)) -
Received request: POST /api/v1/websso
2018-07-18 01:41:08,748 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(130)) - Save in session:
SAML2Client$attemptedAuthentication =
2018-07-18 01:41:08,749 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:set(130)) - Save in session: pac4jUserProfiles =
{SAML2Client=#SAML2Profile# | id: [email protected] | attributes:
{Mail=[[email protected]], notOnOrAfter=2018-07-18T01:46:08.427Z,
sessionindex=X3R7kH9zunD1jn9CP5eN9sm2N0M, notBefore=2018-07-18T01:36:08.427Z} |
roles: [] | permissions: [] | isRemembered: false | clientName: SAML2Client |
linkedId: null |}
2018-07-18 01:41:08,978 ERROR engine.DefaultCallbackLogic
(DefaultCallbackLogic.java:renewSession(123)) - Unable to renew the session.
The session store may not support this feature
2018-07-18 01:41:08,979 DEBUG session.KnoxSessionStore
(KnoxSessionStore.java:get(101)) - Get from session: pac4jRequestedUrl = null
********************
<< gateway-audit.log >>
********************
18/07/18 01:41:06
||be6bf57b-7b96-4292-93ce-00ed574ecd6e|audit|10.89.78.49|YARNUI||||access|uri|/gateway/gate1/yarn/|unavailable|Request
method: GET
18/07/18 01:41:06
|||audit|10.89.78.49|YARNUI||||access|uri|/gateway/gate1/yarn/|success|Response
status: 302
18/07/18 01:41:06
||668bd6c6-664f-499c-97e5-6c2294b98f04|audit|10.89.78.49|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://<dnsname>:8446/gateway/gate1/yarn/|unavailable|Request
method: GET
18/07/18 01:41:06
|||audit|10.89.78.49|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?originalUrl=https://<dnsname>:8446/gateway/gate1/yarn/|success|Response
status: 200
18/07/18 01:41:08
||fdbcaedb-75a6-46a5-8a7d-811f194142d1|audit|10.89.78.49|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client|unavailable|Request
method: POST
18/07/18 01:41:08
|||audit|10.89.78.49|KNOXSSO||||access|uri|/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client|success|Response
status: 302
Thanks,
Praveen.R
From: larry mccay <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Tuesday, July 17, 2018 at 5:56 PM
To: "[email protected]" <[email protected]>
Cc: "[email protected]" <[email protected]>, "Mohanan, Mahesh"
<[email protected]>
Subject: Re: Need help in enabling SAML auth in Apache Knox
Whitelist - this has nothing to do with determining where to redirect - it may
not allow you to redirect somewhere if it doesn't match the expression but it
is not used to determine where to redirect to.
Not sure why the URL would have to be rewritten when proxying.
* try to access YARNUI through Knox
* SSOCookie finds no cookie so redirects to KnoxSSO and provides originalUrl
query param which points to the YARNUI through Knox
* KnoxSSO engages SAML IDP and user logs in
* SAML assertion is POSTED to KnoxSSO endpoint with proper callback URL
* KnoxSSO creates hadoop-jwt cookie and redirects to originalUrl query param
I can't tell if you are saying that the POS to KnoxSSO is going to the wrong
place or the originalUrl redirect is.
On Tue, Jul 17, 2018 at 3:31 PM, Ravikumar, Praveen Krishnamoorthy
<[email protected]<mailto:[email protected]>> wrote:
Thanks a lot for your inputs Sandeep. I genuinely appreciate for your thoughts.
As you mentioned I followed the below docs and enabled SAML. As of now IDP
related issues are fixed,
· Knox is now able to contact the IDP
· POST the SAML request to IDP.
· IDP responds with the window allowing user to provide the credentials.
· Users are getting authenticated.
· I could see the SAML response in the SAML tracer.
·
But after authentication the page is getting redirected to
http://<dnsname>:8446<http://%3cdnsname%3e:8446> and pops message as 404 not
found. I suspect, this is an issue on the knox side.
I have 2 thoughts, either knoxsso.redirect.whitelist.regex param highlighted
below in knoxsso.xml topology is redirecting to invalid URL or do we need to
add any rewrite rules in the application to direct to a valid URL.
Could anyone please help me in this.
KnoxSSO.xml
------------
<topology>
<gateway>
<provider>
<role>federation</role>
<name>pac4j</name>
<enabled>true</enabled>
<param>
<name>pac4j.callbackUrl</name>
<value>https://<dnsName>:8446/gateway/knoxsso/api/v1/websso</value><https://%3cdnsName%3e:8446/gateway/knoxsso/api/v1/websso%3c/value%3e>
</param>
<param>
<name>clientName</name>
<value>SAML2Client</value>
</param>
<param>
<name>saml.identityProviderMetadataPath</name>
<value>/tmp/preprod_metadata_SP.xml</value>
</param>
<param>
<name>saml.serviceProviderMetadataPath</name>
<value>/tmp/preprod_metadata_SP.xml</value>
</param>
<param>
<name>saml.serviceProviderEntityId</name>
<value>https://<dnsName>:8446/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client</value><https://%3cdnsName%3e:8446/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client%3c/value%3e>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
</gateway>
<service>
<role>KNOXSSO</role>
<param>
<name>knoxsso.cookie.secure.only</name>
<value>true</value>
</param>
<param>
<name>knoxsso.token.ttl</name>
<value>100000</value>
</param>
<param>
<name>knoxsso.redirect.whitelist.regex</name>
<value>^https?:\/\/(<dnsName>|localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$</value>
</param>
</service>
</topology>
Thanks,
Praveen.
From: Sandeep Moré <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Monday, July 16, 2018 at 6:25 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: Need help in enabling SAML auth in Apache Knox
Hello Praveen,
Following are some of the blogs that talk about Knox SAML integration
https://cwiki.apache.org/confluence/display/KNOX/KnoxSSO+and+Okta
https://cwiki.apache.org/confluence/display/KNOX/Configuring+Apache+Knox+SSO+with+Ipsilon+using+SAML2
Knox user guide also has some information on how to configure SAML
https://knox.apache.org/books/knox-1-0-0/user-guide.html#Pac4j+Provider+-+CAS+/+OAuth+/+SAML+/+OpenID+Connect
For more details you can take a look at Pac4J provider
https://github.com/pac4j/pac4j/wiki/Clients#saml-support
Now coming back to your options, I could not find info on Ping IDP for SAML so
I cannot comment on the options, this is just my educated guess.
1. Entity ID - I am guessing this is your "saml.serviceProviderEntityId"
value, for me it is
https://www.myhost.com:8443/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client
2. Baseurl - no idea what this is, you probably have to refer to ping IDP
docs
3. ACSURL (where target assertion is Sent) - this is the same as above
for
https://www.myhost.com:8443/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client
On Mon, Jul 16, 2018 at 5:47 PM Ravikumar, Praveen Krishnamoorthy
<[email protected]<mailto:[email protected]>> wrote:
Hi,
Also, Could anyone please help me what should I configure for
1. Entity ID
2. BaseURL (that is used for the entire site)
3. ACSURL (where target assertion is Sent)
On the Identity provider side, to enable SSO in Knox.
Thanks,
Praveen.
From: "Ravikumar, Praveen Krishnamoorthy"
<[email protected]<mailto:[email protected]>>
Date: Monday, July 16, 2018 at 10:36 AM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Need help in enabling SAML auth in Apache Knox
Hi,
I'm Praveen. I'm working on POC to setup Apache Knox on the master node of an
EMR cluster for our client. With the help of documentations I was able to
install KNOX successfully and was able to run few tests. Currently I'm facing
an issue on enabling SAML authentication, which I'm kind of blocked and I don’t
know, how to proceed or troubleshoot the issue. I have provided few details
regarding the issue and I would love to provide more if needed.
Could anyone help me in this, would be very helpful for me to proceed further.
TASK:
-----
To enable SAML authentication for Apache Knox.
NOTE: Apache Knox is installed and running in port 8446
STEP 1: SSO request initiation.
*******************************
- Our client uses PING Federate Identity provider.
- raised a request to register the application for SSO access.
Entity ID -
https://<dnsName>:8446/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client<https://%3cdnsName%3e:8446/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client>
Target URL - https://<dnsName>:8446(I'm<https://%3cdnsName%3e:8446(I'm>
not sure the target URL is valid, I suspect the page is getting redirected to
this link after auth)
- I received a IDP metadata.xml and certificate.
STEP 2: Topology config
***********************
KnoxSSO.xml
------------
<topology>
<gateway>
<provider>
<role>federation</role>
<name>pac4j</name>
<enabled>true</enabled>
<param>
<name>pac4j.callbackUrl</name>
<value>https://<dnsName>:8446/gateway/knoxsso/api/v1/websso</value><https://%3cdnsName%3e:8446/gateway/knoxsso/api/v1/websso%3c/value%3e>
</param>
<param>
<name>clientName</name>
<value>SAML2Client</value>
</param>
<param>
<name>saml.identityProviderMetadataPath</name>
<value>/tmp/preprod_metadata_SP.xml</value>
</param>
<param>
<name>saml.serviceProviderMetadataPath</name>
<value>/tmp/preprod_metadata_SP.xml</value>
</param>
<param>
<name>saml.serviceProviderEntityId</name>
<value>https://<dnsName>:8446/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client</value><https://%3cdnsName%3e:8446/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=SAML2Client%3c/value%3e>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
</gateway>
<service>
<role>KNOXSSO</role>
<param>
<name>knoxsso.cookie.secure.only</name>
<value>true</value>
</param>
<param>
<name>knoxsso.token.ttl</name>
<value>100000</value>
</param>
<param>
<name>knoxsso.redirect.whitelist.regex</name>
<value>^https?:\/\/(emr-knox-webui-dev\.us-west-2\.elb\.amazonaws\.com|localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$</value>
</param>
</service>
</topology>
gate1.xml
---------
<?xml version="1.0" encoding="utf-8"?>
<topology>
<gateway>
<provider>
<role>federation</role>
<name>SSOCookieProvider</name>
<enabled>true</enabled>
<param>
<name>sso.authentication.provider.url</name>
<value>https://<dns-name>:8446/gateway/knoxsso/api/v1/websso</value><https://%3cdns-name%3e:8446/gateway/knoxsso/api/v1/websso%3c/value%3e>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
</gateway>
<service>
<role>YARNUI</role>
<url>http://<dnsname>:8088</url><http://%3cdnsname%3e:8088%3c/url%3e>
</service>
</topology>
PROBLEM:
********
on accessing the YarnUI (firefox browser) after starting the gateway, The
browser gets redirected to the Identity provider URL -> asks for the login
credentials -> on submitting the user is getting authenticated but the
application gets landed to
https://<DNSDomain>:8446<https://%3cDNSDomain%3e:8446> and throws page not
found error.
I'm seeing the SAML request sent and SAML response getting received but it gets
landed to an invalid page after authentication. I'm unable to figure out the
page to land after authentication.
Hope I have provided the required details. please do let me know if you need
any additional details.
Thanks,
Praveen.
