Ok, I'm running out of ideas to debug the issue. No logs like Bosco posted before, and I cannot find a reason for this not to work, when the endpoint is accessible and is correctly set in install.properties.
On Fri, 11 Jan 2019 at 15:42, Odon Copon <odonco...@gmail.com> wrote: > Thanks Bosco, that's really helpful. > Ran the following search "grep -irnw '/' -e 'AuditProviderFactory' > --exclude \*.java --exclude \*.class --exclude \*.jar 2>/dev/null" trying > to look for "AuditProviderFactory" keyword, but there's nothing with that > word in it (exluding java, class and jar files). So no log files like yours. > What could be causing this? I'm restarting the namenode executing the > following command: "sudo /etc/init.d/hadoop-namenode restart", is this > enough or is there any other internal component that needs to be restarted > as well? like the nodemanager per example. > > On Fri, 11 Jan 2019 at 15:34, Don Bosco Durai <bo...@apache.org> wrote: > >> You might want to look into the logs. Here are some of mine from the HDFS >> namenode log .. >> >> >> >> 2019-01-10 06:52:22,128 INFO provider.AuditProviderFactory >> (AuditProviderFactory.java:init(150)) - AUDIT PROPERTY: >> ranger.plugin.hdfs.policy.rest.url=http://orange1.mydomain.com:6080 >> >> >> >> 2019-01-10 06:52:22,200 INFO service.RangerBasePlugin >> (RangerBasePlugin.java:init(151)) - PolicyEngineOptions: { evaluatorType: >> auto, cacheAuditResult: true, disableContextEnrichers: false, >> disableCustomConditions: false, disableTrieLookupPrefilter: false } >> >> 2019-01-10 06:52:23,274 INFO util.PolicyRefresher >> (PolicyRefresher.java:loadPolicyfromPolicyAdmin(277)) - >> PolicyRefresher(serviceName=orange_hadoop): found updated version. >> lastKnownVersion=-1; newVersion=3 >> >> 2019-01-10 06:52:23,337 INFO util.RangerResourceTrie >> (RangerResourceTrie.java:<init>(112)) - resourceName=path; >> optIgnoreCase=false; optWildcard=true; wildcardChars=*?{}\; nodeCount=18; >> leafNodeCount=1; singleChildNodeCount=17; maxDepth=18; >> evaluatorListCount=0; wildcardEvaluatorListCount=2; >> evaluatorListRefCount=17; wildcardEvaluatorListRefCount=15 >> >> >> >> >> >> Bosco >> >> >> >> *From: *Odon Copon <odonco...@gmail.com> >> *Reply-To: *<user@ranger.apache.org> >> *Date: *Friday, January 11, 2019 at 7:23 AM >> *To: *<user@ranger.apache.org> >> *Subject: *Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> On the namenode I'm editing install.properties file and then "sudo >> ./enable-hdfs-plugin.sh". >> >> Then I'm restarting the namenode service. >> >> >> >> On Fri, 11 Jan 2019 at 15:19, Don Bosco Durai <bo...@apache.org> wrote: >> >> It is every 30 seconds. If you are not seeing anything, then it might be >> a configuration issue. How are you enabling the HDFS plugin? >> >> >> >> Bosco >> >> >> >> >> >> *From: *Odon Copon <odonco...@gmail.com> >> *Reply-To: *<user@ranger.apache.org> >> *Date: *Friday, January 11, 2019 at 7:18 AM >> *To: *<user@ranger.apache.org> >> *Subject: *Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> 1.- How often is the plugin pulling policies from the API? >> >> 2.- I don't see anything in the logs regarding the plugin, how can I >> ensure is it running correctly? >> >> 3.- If I run a manual CURL requesting the policies I can see the UI is >> showing that attempt to retrieve the policies, with a 200. But just the >> manual CURL requests, no requests coming from the plugin. >> >> >> >> On Fri, 11 Jan 2019 at 14:58, Don Bosco Durai <bo...@apache.org> wrote: >> >> Also make sure you have the correct hostname and port for Ranger Admin. >> >> >> >> Bosco >> >> >> >> >> >> *From: *Velmurugan Periasamy <vperias...@hortonworks.com> >> *Reply-To: *<user@ranger.apache.org> >> *Date: *Friday, January 11, 2019 at 6:32 AM >> *To: *"user@ranger.apache.org" <user@ranger.apache.org> >> *Subject: *Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> You can check namenode log for any errors from HDFS plugin. >> >> >> ------------------------------ >> >> *From:* Odon Copon <odonco...@gmail.com> >> *Sent:* Friday, January 11, 2019 9:21 AM >> *To:* user@ranger.apache.org >> *Subject:* Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> I fired manually a CURL request to >> "/service/plugins/policies/download/<service_name>" and now the UI is >> displaying some information in plugin tab. >> >> 1. Is Ranger Admin thinking the call was made from the plugin and is >> trying to list it? >> >> 2. If plugin would have executed this request, the UI should have >> displayed this information earlier, right? >> >> 3. Any specific log to check for more information? >> >> >> >> On Fri, 11 Jan 2019 at 14:07, Velmurugan Periasamy < >> vperias...@hortonworks.com> wrote: >> >> You should see plugin sync'ing policies in plugin tab. If it is not >> showing up, you need to check the logs for any error messages. >> ------------------------------ >> >> *From:* Odon Copon <odonco...@gmail.com> >> *Sent:* Friday, January 11, 2019 8:47 AM >> *To:* user@ranger.apache.org >> *Subject:* Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> ok, seems "service/plugins/policies/download/" has public access, so >> confirms what we have been discussing, no authorization is required to >> download the policies. Good to know, thanks guys. >> >> The question know is how to know the plugin is calling this endpoint, and >> working properly, because the UI doesn't display this plugin. Any tip on >> this? >> >> >> >> On Fri, 11 Jan 2019 at 13:08, Odon Copon <odonco...@gmail.com> wrote: >> >> Yes, makes sense to have a 2-way SSL between the plugin and Ranger Admin, >> but: >> >> - 1. Does it mean there's no authentication at all between them? >> >> - 2. If there's no authentication, shouldn't a simple CURL work? At the >> moment if no user/pass is provided the API returns 401, or is there another >> different endpoint? If so, which one is it? >> >> - 3. What is the best way to debug the plugin is communicating or trying >> to communicate with Ranger admin? >> >> >> >> Thanks. >> >> >> >> >> >> On Fri, 11 Jan 2019 at 12:53, Velmurugan Periasamy < >> vperias...@hortonworks.com> wrote: >> >> If there is no kerberos HDFS plugin uses the open Download policies API, >> so it is recommended to use 2-way SSL between HDFS plugin and Ranger Admin. >> >> >> On Jan 11, 2019, at 5:26 AM, Odon Copon <odonco...@gmail.com> wrote: >> >> I cannot perform a CURL to the API from the namenode without >> user/password, I get a 401 when doing that. So it might required >> credentials to do that. If I use the admin/password credentials or >> rangerusersync credentials the CURL works. So wondering if those >> credentials need to be setup somewhere. >> >> >> >> On Fri, 11 Jan 2019 at 10:15, Don Bosco Durai <bo...@apache.org> wrote: >> >> > In terms of "no authentication", is the HDFS plugin using Policy >> Manager API with no credentials at all? >> >> No credentials, because there is no user/password for HDFS service user. >> It’s been a while, I think we used to have admin/password before, but it >> was taken out eventually. The code might be still there… >> >> >> >> > What's the first action the plugin is performing to be detected by the >> UI as active and 200 response? >> >> Abhay or Madhan might be able to give you more specifics. Since the >> plugins are polling and it knows the previous version number, if there are >> no changes, then it is not registered in the UI. The plugins primarily pull >> the policies and tags from Ranger Admin. Rest everything is done by the >> plugin within the component. >> >> >> >> Bosco >> >> >> >> >> >> *From: *Odon Copon <odonco...@gmail.com> >> *Reply-To: *<user@ranger.apache.org> >> *Date: *Friday, January 11, 2019 at 2:03 AM >> *To: *<user@ranger.apache.org> >> *Subject: *Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> Indeed, I know that at the moment without something like Kerberos, users >> can impersonate others, but I'm currently building a POC with the basic >> security to evaluate Ranger, and once is ready, start improving the >> security and scalability. But thank you for pointing that out. >> >> In terms of "no authentication", is the HDFS plugin using Policy Manager >> API with no credentials at all? or default ones? >> >> What's the first action the plugin is performing to be detected by the UI >> as active and 200 response? Some king of ping/heartbeat? or just a rest >> petition to download the policies? >> >> Is there anywhere where I can see in the logs what kind of actions the >> plugin is doing? I don't find any log information coming from the plugin. >> >> >> >> Thanks! >> >> >> >> On Fri, 11 Jan 2019 at 09:53, Don Bosco Durai <bo...@apache.org> wrote: >> >> If there is no Kerberos, then you have 2 options: >> >> 1. No authentication (default) >> 2. Two way SSL to authenticate the request from the plugin. >> >> >> >> Note, if it is non-Kerberos environment, then authorization cannot be >> enforced, because users can impersonate anyone. >> >> >> >> Bosco >> >> >> >> >> >> *From: *Odon Copon <odonco...@gmail.com> >> *Reply-To: *<user@ranger.apache.org> >> *Date: *Friday, January 11, 2019 at 1:22 AM >> *To: *<user@ranger.apache.org> >> *Subject: *Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> Mmm, but what if the system is not using Kerberos? >> >> >> >> On Fri, 11 Jan 2019, 04:21 Velmurugan Periasamy < >> vperias...@hortonworks.com wrote: >> >> Yes, that's what I referred to. >> >> >> ------------------------------ >> >> *From:* Odon Copon <odonco...@gmail.com> >> *Sent:* Thursday, January 10, 2019 5:07 PM >> *To:* user@ranger.apache.org >> *Subject:* Re: Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> Are we talking about principal in Kerberos or any other principal I'm not >> understanding? >> >> >> >> On Thu, 10 Jan 2019 at 18:05, Odon Copon <odonco...@gmail.com> wrote: >> >> What do you mean by *HDFS plugin uses service (Namenode) user's >> principal *? >> >> Could you provide an example? >> Thanks. >> >> >> >> On Thu, 10 Jan 2019 at 17:08, Velmurugan Periasamy < >> vperias...@hortonworks.com> wrote: >> >> HDFS plugin uses service (Namenode) user's principal. >> >> >> ------------------------------ >> >> *From:* Odon Copon <odonco...@gmail.com> >> *Sent:* Thursday, January 10, 2019 8:59 AM >> *To:* user@ranger.apache.org >> *Subject:* Accessing Ranger Policy Manager API from HDFS plugin >> >> >> >> Hi, >> >> How does the Ranger HDFS plugin communicates with the Policy Manager API? >> Is it using a specific user/password combination? >> >> I know the User Sync has rangerusersync user and pass, and all that >> information is stored in rangerusersync.jceks, but what about the HDFS >> plugin or any other plugin? >> I'm having issues with that, my plugin once enabled doesn't get displayed >> in the UI and would like to check the credentials the plugin is using to >> use the API. >> >> For the User Sync - Policy Manager communication works fine. >> >> >> >> Thanks. >> >>
