Rohit, Thanks for sharing your feedback on Ranger integration. Good to know that you are able to get Ranger authorization working in your application.
>> 1. Is it possible to have Ranger running on an unencrypted HDFS >> with secure Hadoop through Kerberos ? Yes. Ranger works with secure Hadoop - either unencrypted or encrypted. >> 2. Currently, I see the following error log for policy cache file. >> Isn’t the policy cache file created automatically ? Yes, the policy cache file is created automatically. I guess the error you see could be cause by non-existent directory where the cache file would be saved - in this case /etc/ranger/myservicedev/policycache/. Please ensure that this directory exists and has write permission for the user that runs the application. The directory location can be specified via configuration ranger.plugin.myservice.policy.cache.dir in ranger-myservice-security.xml. Hope this helps. Madhan On 4/7/16, 7:38 PM, "rohit sinha" <[email protected]> wrote: >Hello, > >Thanks a lot for your prompt replies. It was really helpful. >Yes, it seems like there was some misconfiguration issue. I was able to >make it work and I have a very basic integration up and running with my >service now. I have worked on integration/analysis of other authorization >models and I must say that Ranger so far has been the easiest to integrate >with (at the basic level) and the documentation helped a lot. Appreciate >your efforts in building it so far. > >I have two question which I am trying to figure out. Will appreciate if >someone can provide some pointers. > >1. Kerberos: From the documentation found here >https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-ConfigureKerberosAuthenticationforStorm >I understand that Ranger works with KMS on an encrypted HDFS. Is it >possible to have Ranger running on an unencrypted HDFS with secure Hadoop >through Kerberos ? Something where I can talk to Ranger by giving a >principal and keytab ? > >2. Currently, I see the following error log for policy cache file. Isn't >the policy cache file created automatically ? Am I missing some >configuration. > >16/04/07 21:26:18 INFO util.PolicyRefresher: >> PolicyRefresher(serviceName=myservice): found updated version. >> lastKnownVersion=-1; newVersion=1 >> 16/04/07 21:26:18 ERROR util.PolicyRefresher: failed to save policies to >> cache file '/etc/ranger/myservicedev/policycache/myservice_myservice.json' >> java.io.FileNotFoundException: >> /etc/ranger/myservicedev/policycache/myservice_myservice.json (No such file >> or directory) >> at java.io.FileOutputStream.open(Native Method) >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221) >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171) >> at java.io.FileWriter.<init>(FileWriter.java:90) >> at >> org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:310) >> at >> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191) >> at >> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:134) >> at >> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:105) >> .... internal stacktrace.... >> at java.lang.Thread.run(Thread.java:745) > > > > >Thanks. > >Thanks, >Rohit Sinha > > >On Thu, Apr 7, 2016 at 10:04 AM, Madhan Neethiraj <[email protected]> wrote: > >> Rohit, >> >> To download policies from Ranger Admin, Ranger plugins require the URL to >> Ranger Admin and the name of the service containing the policies. These >> values are read from following configurations from a file named >> ranger-<pluginType>-security.xml >> >> ranger.plugin.<pluginType>.policy.rest.url >> ranger.plugin.<pluginType>.service.name >> >> For example, these are specified in conf/ranger-sampleapp-security.xml for >> the sample application. >> >> Can you please review the configuration for your plugin for the above? >> >> Hope this helps. >> >> Madhan >> >> >> >> On 4/7/16, 5:40 AM, "rohit sinha" <[email protected]> wrote: >> >> >Hello Madhan, >> >Thanks a lot for your reply. >> > >> >I am looking into the integration and I working towards developing the >> >components outside of ranger trunk for initial development purpose. After >> >taking an overview of the codebase it seems that it's possible to have >> >plugins outside of the ranger trunk too. >> > >> >I was able to add my service to Ranger Amin through the curl request and >> >connect to Ranger from my Authorizer but I see the following error: >> > >> >com.sun.jersey.api.client.ClientHandlerException: >> >> java.lang.IllegalArgumentException: URI is not absolute >> >> at >> >> >> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151) >> >> at com.sun.jersey.api.client.Client.handle(Client.java:648) >> >> at >> >> com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) >> >> at >> >> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) >> >> at >> >> com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507) >> >> at >> >> >> org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:94) >> >> at >> >> >> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:215) >> >> at >> >> >> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:183) >> >> at >> >> >> org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:156) >> >> Caused by: java.lang.IllegalArgumentException: URI is not absolute >> >> at java.net.URI.toURL(URI.java:1095) >> >> at >> >> >> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159) >> >> at >> >> >> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) >> >> ... 8 more >> >> 16/04/07 12:31:36 WARN util.PolicyRefresher: cache file does not exist >> or >> >> not readble 'null' >> > >> > >> >A quick search on google pointed towards misconfiguration of service name >> >in security.xml which I have double checked. >> > >> >Any pointers to debug this will be appreciated. >> > >> >Thanks. >> > >> >Thanks, >> >Rohit Sinha >> > >> > >> >On Wed, Apr 6, 2016 at 10:43 PM, Madhan Neethiraj < >> >[email protected]> wrote: >> > >> >> Rohit, >> >> >> >> You are right. REPOSITORY_NAME referenced in the doc is the name of the >> >> service instance in Ranger Admin, which contains the policies for the >> >> component (in this case HBase). The plugin reads this value from a >> >> configuration named ranger.plugin.hbase.service.name (in file >> >> ranger-hbase-security.xml). >> >> >> >> >> After doing this I don't see anything in the Audit -> Plugins >> >> An entry will be created here for every policy download form plugins. >> >> Plugins download the policies at the following events: >> >> - during the component startup (HBase/HiveServer/...) >> >> - when there is a policy change in service instance >> >> >> >> Hope this helps. >> >> >> >> Madhan >> >> >> >> >> >> >> >> On 4/6/16, 7:13 PM, "rohit sinha" <[email protected]> wrote: >> >> >> >> >Thanks for sharing the SampleApp. I was able to run it understand the >> >> >integration point. >> >> >I also tried to enable HBase ranger plugin. When I added the service >> from >> >> >the Ranger Admin UI I was able to talk to HBase and the resource >> >> completion >> >> >worked. After that I deleted the HBase plugin from the UI and tried to >> >> >enable it from the the command line following instructions mentioned >> here: >> >> > >> >> >> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation#ApacheRanger0.5.0Installation-InstallingApacheHBase(1.1.0.1) >> >> > >> >> >After doing this I don't see anything in the Audit -> Plugins. In the >> >> >instructions I do see warning >> >> > >> >> >> Make sure the REPOSITORY_NAME service exists in Ranger Admin. If not, >> >> the >> >> >> hbase-plugin will not be able to communicate with Ranger admin. >> >> > >> >> >Does this mean I need to add the service from the Ranger Admin Panel >> or I >> >> >did something wrong in the enabling the plugin ? >> >> > >> >> >Any help will be highly appreciated. >> >> > >> >> >Thanks. >> >> > >> >> >Thanks, >> >> >Rohit Sinha >> >> > >> >> > >> >> >On Wed, Apr 6, 2016 at 12:43 PM, rohit sinha <[email protected]> >> >> >wrote: >> >> > >> >> >> Awesome. Thanks a lot. >> >> >> >> >> >> Thanks, >> >> >> Rohit Sinha >> >> >> >> >> >> >> >> >> On Wed, Apr 6, 2016 at 12:27 PM, Don Bosco Durai <[email protected]> >> >> wrote: >> >> >> >> >> >>> It is optional. It is easy to setup and helps a lot while debugging >> >> >>> during initial setup. >> >> >>> >> >> >>> Bosco >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> On 4/6/16, 12:23 PM, "rohit sinha" <[email protected]> wrote: >> >> >>> >> >> >>> >Thanks a lot for the prompt replies. Really appreciate it. >> >> >>> >The "Ranger Stacks - How to add a custom plugin?" was really >> helpful >> >> in >> >> >>> >getting some understanding of the integration. I am going through >> the >> >> >>> >SampleApp docs now. >> >> >>> > >> >> >>> >Is auditing an optional feature ? From the documentation it looks >> like >> >> >>> it's >> >> >>> >not and Solr installation is a requirement. I was wondering if I >> can >> >> have >> >> >>> >auditing off and skip Solr installation for initial integration >> >> purpose. >> >> >>> > >> >> >>> >Thanks. >> >> >>> > >> >> >>> >Thanks, >> >> >>> >Rohit Sinha >> >> >>> > >> >> >>> > >> >> >>> >On Wed, Apr 6, 2016 at 11:55 AM, Madhan Neethiraj < >> [email protected]> >> >> >>> wrote: >> >> >>> > >> >> >>> >> Rohit, >> >> >>> >> >> >> >>> >> In addition to the details in the wiki, I would recommend >> reviewing >> >> the >> >> >>> >> following sample application to understand the details of adding >> >> Ranger >> >> >>> >> authorization to an application. >> >> >>> >> >> >> >>> >> - README.txt: >> >> >>> >> >> >> >>> >> >> >> https://github.com/apache/incubator-ranger/blob/master/ranger-examples/README.txt >> >> >>> >> - Application sources: >> >> >>> >> >> >> >>> >> >> >> https://github.com/apache/incubator-ranger/tree/master/ranger-examples/sampleapp >> >> >>> >> . >> >> >>> >> >> >> >>> >> Madhan >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> On 4/6/16, 11:22 AM, "Velmurugan Periasamy" < >> >> >>> [email protected] >> >> >>> >> on behalf of [email protected]> wrote: >> >> >>> >> >> >> >>> >> >Rohit - >> >> >>> >> > >> >> >>> >> >> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741207 >> >> >>> >> >explains how to add a custom plugin for Ranger. >> >> >>> >> > >> >> >>> >> >On 4/6/16, 10:47 AM, "rohit sinha" <[email protected]> >> >> wrote: >> >> >>> >> > >> >> >>> >> >>Hello, >> >> >>> >> >>I am looking into integrating an external service with Apache >> >> Ranger >> >> >>> for >> >> >>> >> >>authorization. >> >> >>> >> >>I looked up the wiki but there is no information about >> integrating >> >> >>> new >> >> >>> >> >>services. >> >> >>> >> >>Can someone give me some info which might be helpful in >> >> identifying >> >> >>> >> >>different components which needs to be developed and other >> >> required >> >> >>> stuff >> >> >>> >> >> >> >> >>> >> >>Thanks. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >>> >> >> >> >> >> >> >>
