Hello Madhan, Thanks a lot for the prompt reply. 1. Can you point me to some resource where I can see how to configure Ranger on Kerberos without KMS.
2. Also, how can I list all the policies for a resource in my service. I don't see any api exposed by RangerBasePlugin for this. I was looking into the code base and saw that ServiceStore has some APIs for this. How can I access this ? Thanks. Thanks, Rohit Sinha On Thu, Apr 7, 2016 at 9:18 PM, Madhan Neethiraj <[email protected]> wrote: > Rohit, > > Thanks for sharing your feedback on Ranger integration. Good to know > that you are able to get Ranger authorization working in your application. > > >> 1. Is it possible to have Ranger running on an unencrypted HDFS > >> with secure Hadoop through Kerberos ? > Yes. Ranger works with secure Hadoop - either unencrypted or encrypted. > > >> 2. Currently, I see the following error log for policy cache file. > >> Isn’t the policy cache file created automatically ? > Yes, the policy cache file is created automatically. I guess the error > > you see could be cause by non-existent directory where the cache file > would be saved - in this case /etc/ranger/myservicedev/policycache/. > Please ensure that this directory exists and has write permission for > the user that runs the application. > > The directory location can be specified via configuration > ranger.plugin.myservice.policy.cache.dir in ranger-myservice-security.xml. > > Hope this helps. > > Madhan > > > > > > On 4/7/16, 7:38 PM, "rohit sinha" <[email protected]> wrote: > > >Hello, > > > >Thanks a lot for your prompt replies. It was really helpful. > >Yes, it seems like there was some misconfiguration issue. I was able to > >make it work and I have a very basic integration up and running with my > >service now. I have worked on integration/analysis of other authorization > >models and I must say that Ranger so far has been the easiest to integrate > >with (at the basic level) and the documentation helped a lot. Appreciate > >your efforts in building it so far. > > > >I have two question which I am trying to figure out. Will appreciate if > >someone can provide some pointers. > > > >1. Kerberos: From the documentation found here > > > https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm > >I understand that Ranger works with KMS on an encrypted HDFS. Is it > >possible to have Ranger running on an unencrypted HDFS with secure Hadoop > >through Kerberos ? Something where I can talk to Ranger by giving a > >principal and keytab ? > > > >2. Currently, I see the following error log for policy cache file. Isn't > >the policy cache file created automatically ? Am I missing some > >configuration. > > > >16/04/07 21:26:18 INFO util.PolicyRefresher: > >> PolicyRefresher(serviceName=myservice): found updated version. > >> lastKnownVersion=-1; newVersion=1 > >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save policies to > >> cache file > '/etc/ranger/myservicedev/policycache/myservice_myservice.json' > >> java.io.FileNotFoundException: > >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No such > file > >> or directory) > >> at java.io.FileOutputStream.open(Native Method) > >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221) > >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171) > >> at java.io.FileWriter.<init>(FileWriter.java:90) > >> at > >> > org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310) > >> at > >> > org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191) > >> at > >> > org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134) > >> at > >> > org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105) > >> .... internal stacktrace.... > >> at java.lang.Thread.run(Thread.java:745) > > > > > > > > > >Thanks. > > > >Thanks, > >Rohit Sinha > > > > > >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <[email protected]> > wrote: > > > >> Rohit, > >> > >> To download policies from Ranger Admin, Ranger plugins require the URL > to > >> Ranger Admin and the name of the service containing the policies. These > >> values are read from following configurations from a file named > >> ranger-<pluginType>-security.xml > >> > >> ranger.plugin.<pluginType>.policy.rest.url > >> ranger.plugin.<pluginType>.service.name > >> > >> For example, these are specified in conf/ranger-sampleapp-security.xml > for > >> the sample application. > >> > >> Can you please review the configuration for your plugin for the above? > >> > >> Hope this helps. > >> > >> Madhan > >> > >> > >> > >> On 4/7/16, 5:40 AM, "rohit sinha" <[email protected]> wrote: > >> > >> >Hello Madhan, > >> >Thanks a lot for your reply. > >> > > >> >I am looking into the integration and I working towards developing the > >> >components outside of ranger trunk for initial development purpose. > After > >> >taking an overview of the codebase it seems that it's possible to have > >> >plugins outside of the ranger trunk too. > >> > > >> >I was able to add my service to Ranger Amin through the curl request > and > >> >connect to Ranger from my Authorizer but I see the following error: > >> > > >> >com.sun.jersey.api.client.ClientHandlerException: > >> >> java.lang.IllegalArgumentException: URI is not absolute > >> >> at > >> >> > >> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151) > >> >> at com.sun.jersey.api.client.Client.handle(Client.java:648) > >> >> at > >> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) > >> >> at > >> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) > >> >> at > >> >> > com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507) > >> >> at > >> >> > >> > org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94) > >> >> at > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215) > >> >> at > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183) > >> >> at > >> >> > >> > org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156) > >> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute > >> >> at java.net.URI.toURL(URI.java:1095) > >> >> at > >> >> > >> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159) > >> >> at > >> >> > >> > com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) > >> >> ... 8 more > >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not > exist > >> or > >> >> not readble 'null' > >> > > >> > > >> >A quick search on google pointed towards misconfiguration of service > name > >> >in security.xml which I have double checked. > >> > > >> >Any pointers to debug this will be appreciated. > >> > > >> >Thanks. > >> > > >> >Thanks, > >> >Rohit Sinha > >> > > >> > > >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj < > >> >[email protected]> wrote: > >> > > >> >> Rohit, > >> >> > >> >> You are right. REPOSITORY_NAME referenced in the doc is the name of > the > >> >> service instance in Ranger Admin, which contains the policies for the > >> >> component (in this case HBase). The plugin reads this value from a > >> >> configuration named ranger.plugin.hbase.service.name (in file > >> >> ranger-hbase-security.xml). > >> >> > >> >> >> After doing this I don't see anything in the Audit -> Plugins > >> >> An entry will be created here for every policy download form plugins. > >> >> Plugins download the policies at the following events: > >> >> - during the component startup (HBase/HiveServer/...) > >> >> - when there is a policy change in service instance > >> >> > >> >> Hope this helps. > >> >> > >> >> Madhan > >> >> > >> >> > >> >> > >> >> On 4/6/16, 7:13 PM, "rohit sinha" <[email protected]> wrote: > >> >> > >> >> >Thanks for sharing the SampleApp. I was able to run it understand > the > >> >> >integration point. > >> >> >I also tried to enable HBase ranger plugin. When I added the service > >> from > >> >> >the Ranger Admin UI I was able to talk to HBase and the resource > >> >> completion > >> >> >worked. After that I deleted the HBase plugin from the UI and tried > to > >> >> >enable it from the the command line following instructions mentioned > >> here: > >> >> > > >> >> > >> > https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1) > >> >> > > >> >> >After doing this I don't see anything in the Audit -> Plugins. In > the > >> >> >instructions I do see warning > >> >> > > >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If > not, > >> >> the > >> >> >> hbase-plugin will not be able to communicate with Ranger admin. > >> >> > > >> >> >Does this mean I need to add the service from the Ranger Admin Panel > >> or I > >> >> >did something wrong in the enabling the plugin ? > >> >> > > >> >> >Any help will be highly appreciated. > >> >> > > >> >> >Thanks. > >> >> > > >> >> >Thanks, > >> >> >Rohit Sinha > >> >> > > >> >> > > >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha < > [email protected]> > >> >> >wrote: > >> >> > > >> >> >> Awesome. Thanks a lot. > >> >> >> > >> >> >> Thanks, > >> >> >> Rohit Sinha > >> >> >> > >> >> >> > >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai < > [email protected]> > >> >> wrote: > >> >> >> > >> >> >>> It is optional. It is easy to setup and helps a lot while > debugging > >> >> >>> during initial setup. > >> >> >>> > >> >> >>> Bosco > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <[email protected]> > wrote: > >> >> >>> > >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it. > >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really > >> helpful > >> >> in > >> >> >>> >getting some understanding of the integration. I am going > through > >> the > >> >> >>> >SampleApp docs now. > >> >> >>> > > >> >> >>> >Is auditing an optional feature ? From the documentation it > looks > >> like > >> >> >>> it's > >> >> >>> >not and Solr installation is a requirement. I was wondering if I > >> can > >> >> have > >> >> >>> >auditing off and skip Solr installation for initial integration > >> >> purpose. > >> >> >>> > > >> >> >>> >Thanks. > >> >> >>> > > >> >> >>> >Thanks, > >> >> >>> >Rohit Sinha > >> >> >>> > > >> >> >>> > > >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj < > >> [email protected]> > >> >> >>> wrote: > >> >> >>> > > >> >> >>> >> Rohit, > >> >> >>> >> > >> >> >>> >> In addition to the details in the wiki, I would recommend > >> reviewing > >> >> the > >> >> >>> >> following sample application to understand the details of > adding > >> >> Ranger > >> >> >>> >> authorization to an application. > >> >> >>> >> > >> >> >>> >> - README.txt: > >> >> >>> >> > >> >> >>> > >> >> > >> > https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt > >> >> >>> >> - Application sources: > >> >> >>> >> > >> >> >>> > >> >> > >> > https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp > >> >> >>> >> . > >> >> >>> >> > >> >> >>> >> Madhan > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" < > >> >> >>> [email protected] > >> >> >>> >> on behalf of [email protected]> wrote: > >> >> >>> >> > >> >> >>> >> >Rohit - > >> >> >>> >> > > >> >> >>> > >> >> > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 > >> >> >>> >> >explains how to add a custom plugin for Ranger. > >> >> >>> >> > > >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <[email protected]> > >> >> wrote: > >> >> >>> >> > > >> >> >>> >> >>Hello, > >> >> >>> >> >>I am looking into integrating an external service with > Apache > >> >> Ranger > >> >> >>> for > >> >> >>> >> >>authorization. > >> >> >>> >> >>I looked up the wiki but there is no information about > >> integrating > >> >> >>> new > >> >> >>> >> >>services. > >> >> >>> >> >>Can someone give me some info which might be helpful in > >> >> identifying > >> >> >>> >> >>different components which needs to be developed and other > >> >> required > >> >> >>> stuff > >> >> >>> >> >> > >> >> >>> >> >>Thanks. > >> >> >>> >> > > >> >> >>> >> > > >> >> >>> >> > > >> >> >>> >> > >> >> >>> >> > >> >> >>> > >> >> >>> > >> >> >> > >> >> > >> > >> > >
