Can you see who is giving 403? Ranger plugin also returns 403 if authorization fails.
You can check Ranger Audit logs Check debug logs of Knox Bosco From: Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Monday, December 21, 2020 at 6:18 AM To: <user@ranger.apache.org> Subject: Re: Method Level Authorization for Knox I made changes and committed my jar files on the server. For the following command: curl -ik -u <user>:<pass> https://<SERVER>:8443/gateway/dsgdev/livy/v1/sessions I am getting the 403 status code: ---------------------------------------------- HTTP/1.1 403 Forbidden Date: Mon, 21 Dec 2020 14:28:17 GMT Set-Cookie: KNOXSESSIONID=node01uq9dlfwvh49d1sczysbwid9wt23.node0;Path=/gateway/dsgdev;Secure;HttpOnly Set-Cookie: rememberMe=deleteMe; Path=/gateway/dsgdev; Max-Age=0; Expires=Sun, 20-Dec-2020 14:28:17 GMT Cache-Control: must-revalidate,no-cache,no-store Content-Type: text/html;charset=iso-8859-1 Content-Length: 348 Server: Jetty(9.4.12.v20180830) <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 403 Forbidden</title> </head> <body><h2>HTTP ERROR 403</h2> <p>Problem accessing /gateway/dsgdev/livy/v1/sessions. Reason: <pre> Forbidden</pre></p><hr><a href="http://eclipse.org/jetty";>Powered by Jetty:// 9.4.12.v20180830</a><hr/> </body> </html> ----------------------------------------------- In the knox's gateway.log the logged value is the following: 2020-12-21 17:58:17,653 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(692)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin On Sat, Dec 19, 2020 at 10:02 PM Don Bosco Durai <bo...@apache.org> wrote: If there are no changes to the method signature, then overwriting the existing jar with the same jar name should work. Bosco From: Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Saturday, December 19, 2020 at 10:30 AM To: <user@ranger.apache.org> Subject: Re: Method Level Authorization for Knox Thanks. I checked the current setup with the LIVYSERVER service and it works fine. Now, I updated the ranger-knox-plugin module and want to copy the generated jar file in the server. I have two questions: 1. Is that ok to copy the jar file to the following directories? /usr/hdp/current/knox-server/ext/ranger-knox-plugin-impl /usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox 2. Is it required to also copy other jar files such as ranger-plugins-common, ranger-plugins-audit to the server? Best On Sun, Dec 13, 2020 at 9:20 PM Don Bosco Durai <bo...@apache.org> wrote: You seemed to getting SSL errors. I will suggest that you try to get the default without your customization working. After that, you can try to putting debug statements at the entry points to make sure you are extracting and passing everything in the Request object. I also assume, you have created the ServiceDef properly. Bosco From: Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Saturday, December 12, 2020 at 11:50 PM To: <user@ranger.apache.org> Subject: Re: Method Level Authorization for Knox Bosco, Thanks for your reply. I followed these steps but could not manage to get it working: 1. I added the GET, DELETE, and POST methods to the service definition's access type and updated the service definition in the server. 2. In the authorization/knox/KnoxRangerPlugin class two methods (actionType and accessType) are added which are respectively used in building the action and access type of the RangerAccessRequest instance. 3. In the RangerPDPKnoxFilter class I extracted the method type from the received ServletRequest and set it as the access type of the RangerAccessRequest. The action type is set to 'allow'. 4. The finally packaged jar file is copied to the following paths in the server: KNOX_SERVER/ext/ranger-knox-plugin-impl RANGER_ADMIN/ews/webapp/WEB-INF/classes/ranger-plugins/knox 5. In the Ambari's KNOX service and in the advanced topology config file I set authorization to XASecurePDPKnox and also added my service to be authorized: <service> <role>MY-SERVICE</role> <url>https://<SERVER>:8443/gateway/dsgdev/livy/v1/sessions</url> </service> 6.In the Ranger UI and for the dsgdev_knox service I added the policy myservice with the topology value of default and knox service value of MY-SERVICE. For the admin user the DELETE permission is set. Here are issues I faced: ٌ 1. When creating the dsgdev_knox service I set the knox.url to https://<server>:8443/gateway/default/api/v1/topologies the connection test is failed: --------------- org.apache.ranger.plugin.client.HadoopException: Exception on REST call to KnoxUrl : https://master01dev.sic.local:8443/gateway/default/api/v1/topologies.. Exception on REST call to KnoxUrl : https://master01dev.sic.local:8443/gateway/default/api/v1/topologies.. java.net.SocketException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext). java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext). Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext). problem accessing trust store. Keystore was tampered with, or password was incorrect. Password verification failed. --------------- 2. When I execute GET on the MY-SERVICE with the admin user since this user only has DELETE permission, the authorization should be failed. BUT authorized. 3. I also got the LookupUser error so had to comment the overrided getDefaultRangerPolicies() method in the RangerServiceKnox class. Sorry for this long description. Thanks in advance for any help Ebrahim On Wed, Dec 9, 2020 at 2:43 PM Don Bosco Durai <bo...@apache.org> wrote: I reviewed the RangerPDPKnoxFilter code. Since Knox has only one resource (topology), you will have to the following: Update the Knox servicedef to add your “model” resource type You have to decide whether you want hierarchy. E.g. topology -> model, model-> topology or model and topology at the same level Update RangerPDPKnoxFilter to create the request with what you want to send to model I might be missing some steps… Thanks Bosco From: Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Tuesday, December 8, 2020 at 10:24 PM To: <user@ranger.apache.org> Subject: Re: Method Level Authorization for Knox Sorry for typos Am I on the right way? On Wed, Dec 9, 2020 at 9:53 AM Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> wrote: Thanks Bosco. What I understand is that the only config we have in the Knox's authorization interface is just set it to XASecurePDPKnox, then the authorization is controlled by the Ranger's Knox plugin. The solution I am working on is to update the knox-agent and the ranger-knox-plugin-shim modules to support the model level authorization. Am I on the write way? Thanks Ebrahim On Tue, Dec 8, 2020 at 1:27 PM Don Bosco Durai <bo...@apache.org> wrote: I think, either will need you to update the Knox’s authorization interface. Please note, Ranger Plugin just implements the interface provided by the host process, in this case Knox. Have you posted the same question the Knox’s mailing list? Thanks Bosco From: Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Tuesday, December 8, 2020 at 1:40 AM To: <user@ranger.apache.org> Subject: Re: Method Level Authorization for Knox There is no suggestion what I should do? On Wed, Dec 2, 2020 at 9:18 AM Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> wrote: Hi, My problem is there. Some suggested me to change the current existing knox plugin and improve it to support the method level authorization, so no integration required. Another proposed solution is to configure the Apache knox so that in addition to the Ranger's knox pluging also use my HTTP service plugin in the chain of authorization process. I am not sure the second solution is easy to implement. On Wed, Dec 2, 2020 at 5:45 AM Velmurugan Periasamy <v...@apache.org> wrote: Hi - can you please elaborate on how you are planning to integrate with the existing Knox plugin? On Tue, Dec 1, 2020 at 12:16 AM Ebrahim Khalil Abbasi <ebrahim.khalil.abb...@gmail.com> wrote: Hi there, I am using knox to access livy to manage spark sessions. To implement authorization I want to provide the method level (get/post/delete/...) authorization. I implemented a new HTTP Service plugin in Ranger but I need to integrate it to the Ranger's knox plugin so that each HTTP request to the knox is authorized based on the method by the Ranger. Thanks for your any help! Ebrahim
