Re: Review Request 71583: RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins for evaluation -part2

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71583/#review218105
---




security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
Line 3412 (original), 3425 (patched)


In addition to policyItems, get roles from denyPolicyItems, allowExceptions 
and denyExceptions as well.

Please review for other such occurances.


- Madhan Neethiraj


On Oct. 7, 2019, 5:07 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71583/
> ---
> 
> (Updated Oct. 7, 2019, 5:07 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2512
> https://issues.apache.org/jira/browse/RANGER-2512
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins 
> for evaluation -part2
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java 9151a72 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> edc886c 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java 0d46ca8 
>   security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java c1ec629 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerRoleService.java 
> e168278 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 1a6b0bd 
> 
> 
> Diff: https://reviews.apache.org/r/71583/diff/3/
> 
> 
> Testing
> ---
> 
> - Addressed review comments in preview patch.
> - "ranger.support.for.service.specific.role.download" introduced to enable 
> role download by service. By default it is "false" and it will download all 
> the roles when add or update of roles happens. If set to "true" only these 
> services which uses the roles will get the updated roles.
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 71583: RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins for evaluation -part2

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71583/
---

(Updated Oct. 7, 2019, 5:07 a.m.)


Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
and Velmurugan Periasamy.


Changes
---

review comments fixed


Bugs: RANGER-2512
https://issues.apache.org/jira/browse/RANGER-2512


Repository: ranger


Description
---

RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins 
for evaluation -part2


Diffs (updated)
-

  security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java 9151a72 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
edc886c 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java 0d46ca8 
  security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java c1ec629 
  security-admin/src/main/java/org/apache/ranger/service/RangerRoleService.java 
e168278 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml 1a6b0bd 


Diff: https://reviews.apache.org/r/71583/diff/3/

Changes: https://reviews.apache.org/r/71583/diff/2-3/


Testing
---

- Addressed review comments in preview patch.
- "ranger.support.for.service.specific.role.download" introduced to enable role 
download by service. By default it is "false" and it will download all the 
roles when add or update of roles happens. If set to "true" only these services 
which uses the roles will get the updated roles.


Thanks,

Ramesh Mani

[jira] [Comment Edited] (RANGER-2510) Support for Incremental tag updates to improve performance

[Abhay Kulkarni (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938056#comment-16938056
 ] 

Abhay Kulkarni edited comment on RANGER-2510 at 10/7/19 3:34 AM:
-

Commit details:

[https://github.com/apache/ranger/commit/c31d4e45281d18b2f18be315d58dc21ea02c7c47]

 

Additional commits:

[https://github.com/apache/ranger/commit/d484e2bad26cc024e36908691138f8c4ac133f47]

 

[https://github.com/apache/ranger/commit/13341c7c22ca78eff2d1eb49bb5b56e8cb68f8d3]


was (Author: abhayk):
Commit details:

[https://github.com/apache/ranger/commit/c31d4e45281d18b2f18be315d58dc21ea02c7c47]

 

Additional commit:

[https://github.com/apache/ranger/commit/d484e2bad26cc024e36908691138f8c4ac133f47]

> Support for Incremental tag updates to improve performance
> --
>
> Key: RANGER-2510
> URL: https://issues.apache.org/jira/browse/RANGER-2510
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: 2.1.0
>
>
> Currently, every change to any tag/service-resource/service-resource->tag 
> mapping causes complete rebuilding of portions of policy-engine that map 
> accessed resource to their tags. There are several disadvantages:
> 1. Compute time for rebuilding
> 2. Large traffic from ranger-admin to each of the plugins
> 3. Large load on JVM memory system because of frequent complete rebuilding of 
> portions of policy-engine.
> It will be more optimal to communicate only the changes to tags and apply 
> them to existing policy-engine.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (RANGER-2510) Support for Incremental tag updates to improve performance

[Abhay Kulkarni (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937977#comment-16937977
 ] 

Abhay Kulkarni edited comment on RANGER-2510 at 10/7/19 3:33 AM:
-

Patch is available at the review board:

[https://reviews.apache.org/r/71542/]

 

[https://reviews.apache.org/r/71584/]


was (Author: abhayk):
Patch is available at the review board:

[https://reviews.apache.org/r/71542/]

> Support for Incremental tag updates to improve performance
> --
>
> Key: RANGER-2510
> URL: https://issues.apache.org/jira/browse/RANGER-2510
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: 2.1.0
>
>
> Currently, every change to any tag/service-resource/service-resource->tag 
> mapping causes complete rebuilding of portions of policy-engine that map 
> accessed resource to their tags. There are several disadvantages:
> 1. Compute time for rebuilding
> 2. Large traffic from ranger-admin to each of the plugins
> 3. Large load on JVM memory system because of frequent complete rebuilding of 
> portions of policy-engine.
> It will be more optimal to communicate only the changes to tags and apply 
> them to existing policy-engine.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 71584: RANGER-2510: Support for Incremental tag updates to improve performance - handle updates to tag policies correctly

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218103
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 6, 2019, 11:07 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> ---
> 
> (Updated Oct. 6, 2019, 11:07 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ensure that policy cache is correctly updated when only resource policy is 
> updated. Also, ensure that updated policy-engine is used for evaluation when 
> policy deltas are used to build policy engine.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
>  5dae0c12b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  576d5e5bb 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  ae88c73ea 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  6cd1df69e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  1325a4020 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java
>  9c50f8a33 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
>  ef5f1d53f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 
> 596f5e841 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  f6beac675 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 51e08e14b 
> 
> 
> Diff: https://reviews.apache.org/r/71584/diff/2/
> 
> 
> Testing
> ---
> 
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71584: RANGER-2510: Support for Incremental tag updates to improve performance - handle updates to tag policies correctly

[Abhay Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/
---

(Updated Oct. 6, 2019, 11:07 p.m.)


Review request for ranger and Madhan Neethiraj.


Changes
---

Addressed review comments


Bugs: RANGER-2510
https://issues.apache.org/jira/browse/RANGER-2510


Repository: ranger


Description
---

Ensure that policy cache is correctly updated when only resource policy is 
updated. Also, ensure that updated policy-engine is used for evaluation when 
policy deltas are used to build policy engine.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
 5dae0c12b 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
 576d5e5bb 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
 ae88c73ea 
  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
 6cd1df69e 
  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
 1325a4020 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java
 9c50f8a33 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java 
ef5f1d53f 
  agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 
596f5e841 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java 
f6beac675 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
51e08e14b 


Diff: https://reviews.apache.org/r/71584/diff/2/

Changes: https://reviews.apache.org/r/71584/diff/1-2/


Testing
---

Tested:
1. policy-cache is correctly updated.
2. Policy evaluation when tag policies are updated.
3. Policy evaluation when tags are updated.


Thanks,

Abhay Kulkarni

Re: Review Request 71583: RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins for evaluation -part2

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71583/#review218102
---


Fix it, then Ship it!





security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
Lines 3473 (patched)


'ret' is already assigned to 'roleNames' in line #3454. So, line #3473 
seems unnecessary. In fact, 'ret' itself is unncessary.


- Madhan Neethiraj


On Oct. 6, 2019, 6:31 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71583/
> ---
> 
> (Updated Oct. 6, 2019, 6:31 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2512
> https://issues.apache.org/jira/browse/RANGER-2512
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins 
> for evaluation -part2
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java 9151a72 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 51e08e1 
>   security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java 0d46ca8 
>   security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java c1ec629 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerRoleService.java 
> e168278 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 1a6b0bd 
> 
> 
> Diff: https://reviews.apache.org/r/71583/diff/2/
> 
> 
> Testing
> ---
> 
> - Addressed review comments in preview patch.
> - "ranger.support.for.service.specific.role.download" introduced to enable 
> role download by service. By default it is "false" and it will download all 
> the roles when add or update of roles happens. If set to "true" only these 
> services which uses the roles will get the updated roles.
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 71584: RANGER-2510: Support for Incremental tag updates to improve performance - handle updates to tag policies correctly

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218101
---




agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
Lines 82 (patched)


It will help to not to make that assumption here. Instead consider the 
following:

  this.policyEngine = policyEngine;

  if (other != null) {
Map localReference = 
other.requestContextEnrichers;

this.rangerPluginContext = other.rangerPluginContext;
this.requestContextEnrichers = MapUtils.isNotEmpty(localReference) ? 
new ConcurrentHashMap<>(localReference) : new ConcurrentHashMap<>();
  } else {
this.rangerPluginContext = null;
this.requestContextEnrichers = new ConcurrentHashMap<>();
  }


- Madhan Neethiraj


On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> ---
> 
> (Updated Oct. 5, 2019, 10:27 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ensure that policy cache is correctly updated when only resource policy is 
> updated. Also, ensure that updated policy-engine is used for evaluation when 
> policy deltas are used to build policy engine.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
>  5dae0c12b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  576d5e5bb 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  ae88c73ea 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  6cd1df69e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  1325a4020 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java
>  9c50f8a33 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
>  ef5f1d53f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 
> 596f5e841 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  f6beac675 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 51e08e14b 
> 
> 
> Diff: https://reviews.apache.org/r/71584/diff/1/
> 
> 
> Testing
> ---
> 
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71584: RANGER-2510: Support for Incremental tag updates to improve performance - handle updates to tag policies correctly

[Abhay Kulkarni]

> On Oct. 6, 2019, 4:13 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
> > Lines 82 (patched)
> > 
> >
> > when 'other' is null but 'policyEngine' is not (refer to call from 
> > RangerPolicyEngineImpl.java #230), this.policyEngine is set to null #92. 
> > This perhaps isn't right? Please review.

The condition that policyEngine not null and other is null will never happen 
when a policy-engine is being constructed using policy-deltas from an existing 
policy-engine. Please refer to (RangerPolicyEngineImpl.java #269). I can take 
out the check in RangerAuthContext.java construction, if that is confusing.


- Abhay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218098
---


On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> ---
> 
> (Updated Oct. 5, 2019, 10:27 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ensure that policy cache is correctly updated when only resource policy is 
> updated. Also, ensure that updated policy-engine is used for evaluation when 
> policy deltas are used to build policy engine.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
>  5dae0c12b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  576d5e5bb 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  ae88c73ea 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  6cd1df69e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  1325a4020 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java
>  9c50f8a33 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
>  ef5f1d53f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 
> 596f5e841 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  f6beac675 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 51e08e14b 
> 
> 
> Diff: https://reviews.apache.org/r/71584/diff/1/
> 
> 
> Testing
> ---
> 
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71584: RANGER-2510: Support for Incremental tag updates to improve performance - handle updates to tag policies correctly

[Abhay Kulkarni]

> On Oct. 6, 2019, 5:26 a.m., Ramesh Mani wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
> > Lines 106 (patched)
> > 
> >
> > Is there a reason why we cannot have setIsShared in in 
> > RangerPolicyEngine Interface?

setIsShared() is an internal housekeeping function, so it not appropriate to 
put it in the interface which others may refer to.


- Abhay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218096
---


On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> ---
> 
> (Updated Oct. 5, 2019, 10:27 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ensure that policy cache is correctly updated when only resource policy is 
> updated. Also, ensure that updated policy-engine is used for evaluation when 
> policy deltas are used to build policy engine.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
>  5dae0c12b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  576d5e5bb 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  ae88c73ea 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  6cd1df69e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  1325a4020 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java
>  9c50f8a33 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
>  ef5f1d53f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 
> 596f5e841 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  f6beac675 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 51e08e14b 
> 
> 
> Diff: https://reviews.apache.org/r/71584/diff/1/
> 
> 
> Testing
> ---
> 
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71583: RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins for evaluation -part2

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71583/
---

(Updated Oct. 6, 2019, 6:31 p.m.)


Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
and Velmurugan Periasamy.


Changes
---

review comments addressed


Bugs: RANGER-2512
https://issues.apache.org/jira/browse/RANGER-2512


Repository: ranger


Description
---

RANGER-2512:RangerRolesRESTClient for serving user group roles to the plugins 
for evaluation -part2


Diffs (updated)
-

  security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java 9151a72 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
51e08e1 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java 0d46ca8 
  security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java c1ec629 
  security-admin/src/main/java/org/apache/ranger/service/RangerRoleService.java 
e168278 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml 1a6b0bd 


Diff: https://reviews.apache.org/r/71583/diff/2/

Changes: https://reviews.apache.org/r/71583/diff/1-2/


Testing (updated)
---

- Addressed review comments in preview patch.
- "ranger.support.for.service.specific.role.download" introduced to enable role 
download by service. By default it is "false" and it will download all the 
roles when add or update of roles happens. If set to "true" only these services 
which uses the roles will get the updated roles.


Thanks,

Ramesh Mani

Re: Review Request 71584: RANGER-2510: Support for Incremental tag updates to improve performance - handle updates to tag policies correctly

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71584/#review218098
---




agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
Lines 82 (patched)


when 'other' is null but 'policyEngine' is not (refer to call from 
RangerPolicyEngineImpl.java #230), this.policyEngine is set to null #92. This 
perhaps isn't right? Please review.


- Madhan Neethiraj


On Oct. 5, 2019, 10:27 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71584/
> ---
> 
> (Updated Oct. 5, 2019, 10:27 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-2510
> https://issues.apache.org/jira/browse/RANGER-2510
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ensure that policy cache is correctly updated when only resource policy is 
> updated. Also, ensure that updated policy-engine is used for evaluation when 
> policy deltas are used to build policy engine.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineCache.java
>  5dae0c12b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  576d5e5bb 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  ae88c73ea 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  6cd1df69e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  1325a4020 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPolicyDeltaUtil.java
>  9c50f8a33 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
>  ef5f1d53f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServiceDefUtil.java 
> 596f5e841 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  f6beac675 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 51e08e14b 
> 
> 
> Diff: https://reviews.apache.org/r/71584/diff/1/
> 
> 
> Testing
> ---
> 
> Tested:
> 1. policy-cache is correctly updated.
> 2. Policy evaluation when tag policies are updated.
> 3. Policy evaluation when tags are updated.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 71585: RANGER-2605 : Update Maven Version

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71585/#review218097
---




pom.xml
Line 72 (original), 72 (patched)


3.6.2 is the latest Maven version available. Please consider moving to this 
version.


- Madhan Neethiraj


On Oct. 6, 2019, 2:27 p.m., Velmurugan Periasamy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71585/
> ---
> 
> (Updated Oct. 6, 2019, 2:27 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-2605
> https://issues.apache.org/jira/browse/RANGER-2605
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2605 - maven version needs to be updated to 3.3.9
> 
> 
> Diffs
> -
> 
>   pom.xml bb2e84728 
> 
> 
> Diff: https://reviews.apache.org/r/71585/diff/1/
> 
> 
> Testing
> ---
> 
> Built successfully with unit tests passing
> 
> 
> Thanks,
> 
> Velmurugan Periasamy
> 
>

[jira] [Commented] (RANGER-2605) Update maven version to 3.3.9

[Velmurugan Periasamy (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945360#comment-16945360
 ] 

Velmurugan Periasamy commented on RANGER-2605:
--

Review available at https://reviews.apache.org/r/71585

> Update maven version to 3.3.9
> -
>
> Key: RANGER-2605
> URL: https://issues.apache.org/jira/browse/RANGER-2605
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Velmurugan Periasamy
>Priority: Major
> Fix For: 2.1.0
>
>
> Maven version needs to be updated to 3.3.9. See 
> https://issues.apache.org/jira/browse/INFRA-19222



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2605) Update maven version to 3.3.9

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2605:
-
Fix Version/s: 2.1.0

> Update maven version to 3.3.9
> -
>
> Key: RANGER-2605
> URL: https://issues.apache.org/jira/browse/RANGER-2605
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Velmurugan Periasamy
>Priority: Major
> Fix For: 2.1.0
>
>
> Maven version needs to be updated to 3.3.9. See 
> https://issues.apache.org/jira/browse/INFRA-19222



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 71585: RANGER-2605 : Update Maven Version

[Velmurugan Periasamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71585/
---

Review request for ranger.


Bugs: RANGER-2605
https://issues.apache.org/jira/browse/RANGER-2605


Repository: ranger


Description
---

RANGER-2605 - maven version needs to be updated to 3.3.9


Diffs
-

  pom.xml bb2e84728 


Diff: https://reviews.apache.org/r/71585/diff/1/


Testing
---

Built successfully with unit tests passing


Thanks,

Velmurugan Periasamy

[jira] [Updated] (RANGER-2605) Update maven version to 3.3.9

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2605:
-
Summary: Update maven version to 3.3.9  (was: Update maven version )

> Update maven version to 3.3.9
> -
>
> Key: RANGER-2605
> URL: https://issues.apache.org/jira/browse/RANGER-2605
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Velmurugan Periasamy
>Priority: Major
>
> Maven version needs to be updated to 3.3.9. See 
> https://issues.apache.org/jira/browse/INFRA-19222



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2605) Update maven version

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2605:
-
Description: Maven version needs to be updated to 3.3.9. See 
https://issues.apache.org/jira/browse/INFRA-19222

> Update maven version 
> -
>
> Key: RANGER-2605
> URL: https://issues.apache.org/jira/browse/RANGER-2605
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Velmurugan Periasamy
>Priority: Major
>
> Maven version needs to be updated to 3.3.9. See 
> https://issues.apache.org/jira/browse/INFRA-19222



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (RANGER-2605) Update maven version

[Velmurugan Periasamy (Jira)]

Velmurugan Periasamy created RANGER-2605:


 Summary: Update maven version 
 Key: RANGER-2605
 URL: https://issues.apache.org/jira/browse/RANGER-2605
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Velmurugan Periasamy
Assignee: Velmurugan Periasamy






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2604) Can't connect to Presto Pugin when TLS is enabled on Presto

[David Berger (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Berger updated RANGER-2604:
-
Description: 
We are running Presto with TLS enabled 
[https://prestosql.github.io/docs.prestosql.io/current/security/tls.html#server-java-keystore]

 

When connecting to Presto via a JDBC client it works fine by enabling SSL and 
passing the trust store details like below

jdbc:presto://edl-hr-pr-ldap-presto.az.gdp-bigdata1.gdpdentsu.net:443/hive/default?SSL=true=/Users/david.berger/git/tactical-edl-hr/presto/edl-hr-keystore-coordinator_trust.jks=turstpass123

 

But using the same connection string when setting up the Presto Repo in Ranger 
it doesn't work because Ranger assumes you're running Kerberos now, which isn't 
right.

 

*See the Ranger REST call we use to create the repo below:*

curl -iv -u ${RANGER_ADMIN_USER}:${RANGER_ADMIN_PWD} -H "Content-Type: 
application/json" -d '{"configs":

{"username": "LDAPADM", "password": "", "jdbc.driverClassName": 
"io.prestosql.jdbc.PrestoDriver", "jdbc.url": 
"jdbc:presto://edl-hr-pr-ldap-presto.az.gdp-bigdata1.gdpdentsu.net:443/hive/default?SSL=true=/plugins_tls/edl-hr-keystore-coordinator_trust.jks=turstpass123"}

, "description": "PrestoTestRepo", "isEnabled": true, "name": "PrestoTestRepo", 
"type": "presto", "version": 1 }' -X POST ${URL}/service/public/v2/api/service

 

*The error in the Ranger log preventing us from logging in:*

2019-10-06 07:47:44,562 [timed-executor-pool-0] WARN 
org.apache.hadoop.security.SecureClientLogin (SecureClientLogin.java:126) - 
Can't find keyTab Path : null
2019-10-06 07:47:44,562 [timed-executor-pool-0] WARN 
org.apache.hadoop.security.SecureClientLogin (SecureClientLogin.java:130) - 
Can't find principal : null
2019-10-06 07:47:44,567 [timed-executor-pool-0] INFO 
org.apache.ranger.plugin.client.BaseClient (BaseClient.java:126) - Init Login: 
security not enabled, using username

 

  was:
We are running Presto with TLS enabled 
[https://prestosql.github.io/docs.prestosql.io/current/security/tls.html#server-java-keystore]

 

When connecting to Presto via a JDBC client it works fine by enabling SSL and 
passing the trust store details like below

jdbc:presto://edl-hr-pr-ldap-presto.az.gdp-bigdata1.gdpdentsu.net:443/hive/default?SSL=true=/Users/david.berger/git/tactical-edl-hr/presto/edl-hr-keystore-coordinator_trust.jks=turstpass123

 

But using the same connection string when setting up the Presto Repo in Ranger 
it doesn't work because Ranger assumes you're running Kerberos now, which isn't 
right.

 

*See the Ranger REST call we use to create the repo below:*

curl -iv -u ${RANGER_ADMIN_USER}:${RANGER_ADMIN_PWD} -H "Content-Type: 
application/json" -d '\{"configs": {"username": "LDAPADM", "password": 
"", "jdbc.driverClassName": "io.prestosql.jdbc.PrestoDriver", 
"jdbc.url": 
"jdbc:presto://edl-hr-pr-ldap-presto.az.gdp-bigdata1.gdpdentsu.net:443/hive/default?SSL=true=/plugins_tls/edl-hr-keystore-coordinator_trust.jks=turstpass123"},
 "description": "PrestoTestRepo", "isEnabled": true, "name": "PrestoTestRepo", 
"type": "presto", "version": 1 }' -X POST ${URL}/service/public/v2/api/service

 

*The error in the Ranger log preventing us from logging in:*

019-10-06 07:47:44,562 [timed-executor-pool-0] WARN  
org.apache.hadoop.security.SecureClientLogin (SecureClientLogin.java:126) - 
*Can't find keyTab Path : null*019-10-06 07:47:44,562 [timed-executor-pool-0] 
WARN  org.apache.hadoop.security.SecureClientLogin (SecureClientLogin.java:126) 
- *Can't find keyTab Path : null*2019-10-06 07:47:44,562 
[timed-executor-pool-0] WARN  org.apache.hadoop.security.SecureClientLogin 
(SecureClientLogin.java:130) - Can't find principal : null2019-10-06 
07:47:44,567 [timed-executor-pool-0] INFO  
org.apache.ranger.plugin.client.BaseClient (BaseClient.java:126) - Init Login: 
security not enabled, using username2019-10-06 07:47:46,716 
[timed-executor-pool-0] ERROR 
apache.ranger.services.presto.client.PrestoClient$2 (PrestoClient.java:213) - 
<== PrestoClient getCatalogList() :Unable to get the Database 
Listorg.apache.ranger.plugin.client.HadoopException: Unable to execute SQL 
[SHOW CATALOGS]. at 
org.apache.ranger.services.presto.client.PrestoClient.getCatalogs(PrestoClient.java:190)
 at 
org.apache.ranger.services.presto.client.PrestoClient.access$100(PrestoClient.java:45)
 at 
org.apache.ranger.services.presto.client.PrestoClient$2.run(PrestoClient.java:211)
 at 
org.apache.ranger.services.presto.client.PrestoClient$2.run(PrestoClient.java:206)
 at java.security.AccessController.doPrivileged(Native Method) at 
javax.security.auth.Subject.doAs(Subject.java:360) at 
org.apache.ranger.services.presto.client.PrestoClient.getCatalogList(PrestoClient.java:206)
 at 
org.apache.ranger.services.presto.client.PrestoClient.connectionTest(PrestoClient.java:497)
 at 

[jira] [Created] (RANGER-2604) Can't connect to Presto Pugin when TLS is enabled on Presto

[David Berger (Jira)]

David Berger created RANGER-2604:


 Summary: Can't connect to Presto Pugin when TLS is enabled on 
Presto
 Key: RANGER-2604
 URL: https://issues.apache.org/jira/browse/RANGER-2604
 Project: Ranger
  Issue Type: Bug
  Components: plugins
Affects Versions: 2.0.0
Reporter: David Berger


We are running Presto with TLS enabled 
[https://prestosql.github.io/docs.prestosql.io/current/security/tls.html#server-java-keystore]

 

When connecting to Presto via a JDBC client it works fine by enabling SSL and 
passing the trust store details like below

jdbc:presto://edl-hr-pr-ldap-presto.az.gdp-bigdata1.gdpdentsu.net:443/hive/default?SSL=true=/Users/david.berger/git/tactical-edl-hr/presto/edl-hr-keystore-coordinator_trust.jks=turstpass123

 

But using the same connection string when setting up the Presto Repo in Ranger 
it doesn't work because Ranger assumes you're running Kerberos now, which isn't 
right.

 

*See the Ranger REST call we use to create the repo below:*

curl -iv -u ${RANGER_ADMIN_USER}:${RANGER_ADMIN_PWD} -H "Content-Type: 
application/json" -d '\{"configs": {"username": "LDAPADM", "password": 
"", "jdbc.driverClassName": "io.prestosql.jdbc.PrestoDriver", 
"jdbc.url": 
"jdbc:presto://edl-hr-pr-ldap-presto.az.gdp-bigdata1.gdpdentsu.net:443/hive/default?SSL=true=/plugins_tls/edl-hr-keystore-coordinator_trust.jks=turstpass123"},
 "description": "PrestoTestRepo", "isEnabled": true, "name": "PrestoTestRepo", 
"type": "presto", "version": 1 }' -X POST ${URL}/service/public/v2/api/service

 

*The error in the Ranger log preventing us from logging in:*

019-10-06 07:47:44,562 [timed-executor-pool-0] WARN  
org.apache.hadoop.security.SecureClientLogin (SecureClientLogin.java:126) - 
*Can't find keyTab Path : null*019-10-06 07:47:44,562 [timed-executor-pool-0] 
WARN  org.apache.hadoop.security.SecureClientLogin (SecureClientLogin.java:126) 
- *Can't find keyTab Path : null*2019-10-06 07:47:44,562 
[timed-executor-pool-0] WARN  org.apache.hadoop.security.SecureClientLogin 
(SecureClientLogin.java:130) - Can't find principal : null2019-10-06 
07:47:44,567 [timed-executor-pool-0] INFO  
org.apache.ranger.plugin.client.BaseClient (BaseClient.java:126) - Init Login: 
security not enabled, using username2019-10-06 07:47:46,716 
[timed-executor-pool-0] ERROR 
apache.ranger.services.presto.client.PrestoClient$2 (PrestoClient.java:213) - 
<== PrestoClient getCatalogList() :Unable to get the Database 
Listorg.apache.ranger.plugin.client.HadoopException: Unable to execute SQL 
[SHOW CATALOGS]. at 
org.apache.ranger.services.presto.client.PrestoClient.getCatalogs(PrestoClient.java:190)
 at 
org.apache.ranger.services.presto.client.PrestoClient.access$100(PrestoClient.java:45)
 at 
org.apache.ranger.services.presto.client.PrestoClient$2.run(PrestoClient.java:211)
 at 
org.apache.ranger.services.presto.client.PrestoClient$2.run(PrestoClient.java:206)
 at java.security.AccessController.doPrivileged(Native Method) at 
javax.security.auth.Subject.doAs(Subject.java:360) at 
org.apache.ranger.services.presto.client.PrestoClient.getCatalogList(PrestoClient.java:206)
 at 
org.apache.ranger.services.presto.client.PrestoClient.connectionTest(PrestoClient.java:497)
 at 
org.apache.ranger.services.presto.client.PrestoResourceManager.connectionTest(PrestoResourceManager.java:48)
 at 
org.apache.ranger.services.presto.RangerServicePresto.validateConfig(RangerServicePresto.java:48)
 at 
org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:660)
 at 
org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:647)
 at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:608) at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.java:748)Caused by: java.sql.SQLException: 
Authentication failed: Access Denied: Invalid credentials at 
io.prestosql.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:271) at 
io.prestosql.jdbc.PrestoStatement.execute(PrestoStatement.java:227) at 
io.prestosql.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:76) at 
org.apache.ranger.services.presto.client.PrestoClient.getCatalogs(PrestoClient.java:173)
 ... 16 moreCaused by: io.prestosql.jdbc.$internal.client.ClientException: 
Authentication failed: Access Denied: Invalid credentials at 
io.prestosql.jdbc.$internal.client.StatementClientV1.requestFailedException(StatementClientV1.java:459)
 at 
io.prestosql.jdbc.$internal.client.StatementClientV1.(StatementClientV1.java:135)
 at 
io.prestosql.jdbc.$internal.client.StatementClientFactory.newStatementClient(StatementClientFactory.java:24)
 at io.prestosql.jdbc.QueryExecutor.startQuery(QueryExecutor.java:46) at