Hello Madhan, I am having some difficulty understanding how ranger and services identify each other on a secure cluster.
For example, if I have a resource lookup plugin running in ranger for HBase and a ranger authorization plugin running in HBase and when they make request to each other how does ranger make sure that the talking service in HBase and also how does HBase identifies the talking service in ranger. Also, for every service we take username and password while the service is being added. What is it used for ? Thanks. Thanks, Rohit Sinha On Fri, Apr 8, 2016 at 4:38 PM, Madhan Neethiraj <[email protected]> wrote: > Rohit, > > >> 1. Can you point me to some resource where I can see how to configure > Ranger on Kerberos without KMS. > Ranger configuration does not depend on presence or absence of KMS. It will > help if you can provide more details on what you are trying to do or what > difference/difficulty you see due to presence/absence of KMS. > > >> 2. Also, how can I list all the policies for a resource in my service > > > > RangerBasePlugin is designed for authorizing accesses and don’t provide > APIs to > search the policies. You can use REST APIs of Ranger Admin to search > policies. > > > For example: curl -f -X GET -H "Accept: application/json" -u > admin-user:admin-pass > http://ranger-admin-host:6080/service/public/v2/api/service/cl1_hive/policy?resource:table=employee1 > > For details of the REST APIs, please refer to: > https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management > > Thanks, > Madhan > > On 4/7/16, 9:28 PM, "rohit sinha" <[email protected]> wrote: > > >Hello Madhan, > >Thanks a lot for the prompt reply. > > > >1. Can you point me to some resource where I can see how to configure > >Ranger on Kerberos without KMS. > > > >2. Also, how can I list all the policies for a resource in my service. I > >don't see any api exposed by RangerBasePlugin for this. I was looking into > >the code base and saw that ServiceStore has some APIs for this. How can I > >access this ? > > > >Thanks. > > > >Thanks, > >Rohit Sinha > > > > > >On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <[email protected]> > wrote: > > > >> Rohit, > >> > >> Thanks for sharing your feedback on Ranger integration. Good to know > >> that you are able to get Ranger authorization working in your > application. > >> > >> >> 1. Is it possible to have Ranger running on an unencrypted HDFS > >> >> with secure Hadoop through Kerberos ? > >> Yes. Ranger works with secure Hadoop - either unencrypted or encrypted. > >> > >> >> 2. Currently, I see the following error log for policy cache file. > >> >> Isn’t the policy cache file created automatically ? > >> Yes, the policy cache file is created automatically. I guess the error > >> > >> you see could be cause by non-existent directory where the cache file > >> would be saved - in this case /etc/ranger/myservicedev/policycache/. > >> Please ensure that this directory exists and has write permission for > >> the user that runs the application. > >> > >> The directory location can be specified via configuration > >> ranger.plugin.myservice.policy.cache.dir in > ranger-myservice-security.xml. > >> > >> Hope this helps. > >> > >> Madhan > >> > >> > >> > >> > >> > >> On 4/7/16, 7:38 PM, "rohit sinha" <[email protected]> wrote: > >> > >> >Hello, > >> > > >> >Thanks a lot for your prompt replies. It was really helpful. > >> >Yes, it seems like there was some misconfiguration issue. I was able to > >> >make it work and I have a very basic integration up and running with my > >> >service now. I have worked on integration/analysis of other > authorization > >> >models and I must say that Ranger so far has been the easiest to > integrate > >> >with (at the basic level) and the documentation helped a lot. > Appreciate > >> >your efforts in building it so far. > >> > > >> >I have two question which I am trying to figure out. Will appreciate if > >> >someone can provide some pointers. > >> > > >> >1. Kerberos: From the documentation found here > >> > > >> > https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm > >> >I understand that Ranger works with KMS on an encrypted HDFS. Is it > >> >possible to have Ranger running on an unencrypted HDFS with secure > Hadoop > >> >through Kerberos ? Something where I can talk to Ranger by giving a > >> >principal and keytab ? > >> > > >> >2. Currently, I see the following error log for policy cache file. > Isn't > >> >the policy cache file created automatically ? Am I missing some > >> >configuration. > >> > > >> >16/04/07 21:26:18 INFO util.PolicyRefresher: > >> >> PolicyRefresher(serviceName=myservice): found updated version. > >> >> lastKnownVersion=-1; newVersion=1 > >> >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save > policies to > >> >> cache file > >> '/etc/ranger/myservicedev/policycache/myservice_myservice.json' > >> >> java.io.FileNotFoundException: > >> >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No > such > >> file > >> >> or directory) > >> >> at java.io.FileOutputStream.open(Native Method) > >> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221) > >> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171) > >> >> at java.io.FileWriter.<init>(FileWriter.java:90) > >> >> at > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310) > >> >> at > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191) > >> >> at > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134) > >> >> at > >> >> > >> > org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105) > >> >> .... internal stacktrace.... > >> >> at java.lang.Thread.run(Thread.java:745) > >> > > >> > > >> > > >> > > >> >Thanks. > >> > > >> >Thanks, > >> >Rohit Sinha > >> > > >> > > >> >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <[email protected]> > >> wrote: > >> > > >> >> Rohit, > >> >> > >> >> To download policies from Ranger Admin, Ranger plugins require the > URL > >> to > >> >> Ranger Admin and the name of the service containing the policies. > These > >> >> values are read from following configurations from a file named > >> >> ranger-<pluginType>-security.xml > >> >> > >> >> ranger.plugin.<pluginType>.policy.rest.url > >> >> ranger.plugin.<pluginType>.service.name > >> >> > >> >> For example, these are specified in > conf/ranger-sampleapp-security.xml > >> for > >> >> the sample application. > >> >> > >> >> Can you please review the configuration for your plugin for the > above? > >> >> > >> >> Hope this helps. > >> >> > >> >> Madhan > >> >> > >> >> > >> >> > >> >> On 4/7/16, 5:40 AM, "rohit sinha" <[email protected]> wrote: > >> >> > >> >> >Hello Madhan, > >> >> >Thanks a lot for your reply. > >> >> > > >> >> >I am looking into the integration and I working towards developing > the > >> >> >components outside of ranger trunk for initial development purpose. > >> After > >> >> >taking an overview of the codebase it seems that it's possible to > have > >> >> >plugins outside of the ranger trunk too. > >> >> > > >> >> >I was able to add my service to Ranger Amin through the curl request > >> and > >> >> >connect to Ranger from my Authorizer but I see the following error: > >> >> > > >> >> >com.sun.jersey.api.client.ClientHandlerException: > >> >> >> java.lang.IllegalArgumentException: URI is not absolute > >> >> >> at > >> >> >> > >> >> > >> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151) > >> >> >> at > com.sun.jersey.api.client.Client.handle(Client.java:648) > >> >> >> at > >> >> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) > >> >> >> at > >> >> >> > com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) > >> >> >> at > >> >> >> > >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507) > >> >> >> at > >> >> >> > >> >> > >> > org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94) > >> >> >> at > >> >> >> > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215) > >> >> >> at > >> >> >> > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183) > >> >> >> at > >> >> >> > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156) > >> >> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute > >> >> >> at java.net.URI.toURL(URI.java:1095) > >> >> >> at > >> >> >> > >> >> > >> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159) > >> >> >> at > >> >> >> > >> >> > >> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) > >> >> >> ... 8 more > >> >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not > >> exist > >> >> or > >> >> >> not readble 'null' > >> >> > > >> >> > > >> >> >A quick search on google pointed towards misconfiguration of service > >> name > >> >> >in security.xml which I have double checked. > >> >> > > >> >> >Any pointers to debug this will be appreciated. > >> >> > > >> >> >Thanks. > >> >> > > >> >> >Thanks, > >> >> >Rohit Sinha > >> >> > > >> >> > > >> >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj < > >> >> >[email protected]> wrote: > >> >> > > >> >> >> Rohit, > >> >> >> > >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the name > of > >> the > >> >> >> service instance in Ranger Admin, which contains the policies for > the > >> >> >> component (in this case HBase). The plugin reads this value from a > >> >> >> configuration named ranger.plugin.hbase.service.name (in file > >> >> >> ranger-hbase-security.xml). > >> >> >> > >> >> >> >> After doing this I don't see anything in the Audit -> Plugins > >> >> >> An entry will be created here for every policy download form > plugins. > >> >> >> Plugins download the policies at the following events: > >> >> >> - during the component startup (HBase/HiveServer/...) > >> >> >> - when there is a policy change in service instance > >> >> >> > >> >> >> Hope this helps. > >> >> >> > >> >> >> Madhan > >> >> >> > >> >> >> > >> >> >> > >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <[email protected]> > wrote: > >> >> >> > >> >> >> >Thanks for sharing the SampleApp. I was able to run it understand > >> the > >> >> >> >integration point. > >> >> >> >I also tried to enable HBase ranger plugin. When I added the > service > >> >> from > >> >> >> >the Ranger Admin UI I was able to talk to HBase and the resource > >> >> >> completion > >> >> >> >worked. After that I deleted the HBase plugin from the UI and > tried > >> to > >> >> >> >enable it from the the command line following instructions > mentioned > >> >> here: > >> >> >> > > >> >> >> > >> >> > >> > https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1) > >> >> >> > > >> >> >> >After doing this I don't see anything in the Audit -> Plugins. In > >> the > >> >> >> >instructions I do see warning > >> >> >> > > >> >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. > If > >> not, > >> >> >> the > >> >> >> >> hbase-plugin will not be able to communicate with Ranger admin. > >> >> >> > > >> >> >> >Does this mean I need to add the service from the Ranger Admin > Panel > >> >> or I > >> >> >> >did something wrong in the enabling the plugin ? > >> >> >> > > >> >> >> >Any help will be highly appreciated. > >> >> >> > > >> >> >> >Thanks. > >> >> >> > > >> >> >> >Thanks, > >> >> >> >Rohit Sinha > >> >> >> > > >> >> >> > > >> >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha < > >> [email protected]> > >> >> >> >wrote: > >> >> >> > > >> >> >> >> Awesome. Thanks a lot. > >> >> >> >> > >> >> >> >> Thanks, > >> >> >> >> Rohit Sinha > >> >> >> >> > >> >> >> >> > >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai < > >> [email protected]> > >> >> >> wrote: > >> >> >> >> > >> >> >> >>> It is optional. It is easy to setup and helps a lot while > >> debugging > >> >> >> >>> during initial setup. > >> >> >> >>> > >> >> >> >>> Bosco > >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > >> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <[email protected]> > >> wrote: > >> >> >> >>> > >> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it. > >> >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really > >> >> helpful > >> >> >> in > >> >> >> >>> >getting some understanding of the integration. I am going > >> through > >> >> the > >> >> >> >>> >SampleApp docs now. > >> >> >> >>> > > >> >> >> >>> >Is auditing an optional feature ? From the documentation it > >> looks > >> >> like > >> >> >> >>> it's > >> >> >> >>> >not and Solr installation is a requirement. I was wondering > if I > >> >> can > >> >> >> have > >> >> >> >>> >auditing off and skip Solr installation for initial > integration > >> >> >> purpose. > >> >> >> >>> > > >> >> >> >>> >Thanks. > >> >> >> >>> > > >> >> >> >>> >Thanks, > >> >> >> >>> >Rohit Sinha > >> >> >> >>> > > >> >> >> >>> > > >> >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj < > >> >> [email protected]> > >> >> >> >>> wrote: > >> >> >> >>> > > >> >> >> >>> >> Rohit, > >> >> >> >>> >> > >> >> >> >>> >> In addition to the details in the wiki, I would recommend > >> >> reviewing > >> >> >> the > >> >> >> >>> >> following sample application to understand the details of > >> adding > >> >> >> Ranger > >> >> >> >>> >> authorization to an application. > >> >> >> >>> >> > >> >> >> >>> >> - README.txt: > >> >> >> >>> >> > >> >> >> >>> > >> >> >> > >> >> > >> > https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt > >> >> >> >>> >> - Application sources: > >> >> >> >>> >> > >> >> >> >>> > >> >> >> > >> >> > >> > https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp > >> >> >> >>> >> . > >> >> >> >>> >> > >> >> >> >>> >> Madhan > >> >> >> >>> >> > >> >> >> >>> >> > >> >> >> >>> >> > >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" < > >> >> >> >>> [email protected] > >> >> >> >>> >> on behalf of [email protected]> wrote: > >> >> >> >>> >> > >> >> >> >>> >> >Rohit - > >> >> >> >>> >> > > >> >> >> >>> > >> >> >> > >> >> > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 > >> >> >> >>> >> >explains how to add a custom plugin for Ranger. > >> >> >> >>> >> > > >> >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" < > [email protected]> > >> >> >> wrote: > >> >> >> >>> >> > > >> >> >> >>> >> >>Hello, > >> >> >> >>> >> >>I am looking into integrating an external service with > >> Apache > >> >> >> Ranger > >> >> >> >>> for > >> >> >> >>> >> >>authorization. > >> >> >> >>> >> >>I looked up the wiki but there is no information about > >> >> integrating > >> >> >> >>> new > >> >> >> >>> >> >>services. > >> >> >> >>> >> >>Can someone give me some info which might be helpful in > >> >> >> identifying > >> >> >> >>> >> >>different components which needs to be developed and > other > >> >> >> required > >> >> >> >>> stuff > >> >> >> >>> >> >> > >> >> >> >>> >> >>Thanks. > >> >> >> >>> >> > > >> >> >> >>> >> > > >> >> >> >>> >> > > >> >> >> >>> >> > >> >> >> >>> >> > >> >> >> >>> > >> >> >> >>> > >> >> >> >> > >> >> >> > >> >> > >> >> > >> > >> > >
