Re: Review Request 56640: Support for using resource-matcher for filtering policies within a service if service-resource is provided in the filter

2017-02-23 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
---

(Updated Feb. 23, 2017, 9:56 p.m.)


Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Changes
---

Addressed review comments.


Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383


Repository: ranger


Description
---

Ranger admin's REST API support retrieving and filtering policies for resource 
specified in the provided filter. Currently, a simple string-match and 
wildcard-match is used to filter policies. It is desirable to provide an option 
to use, for filtering purpose, the same resource-matching algorithm that is 
used by the policy engine to search policies that need to be evaluated for 
access determination in the component.

A new option ("resourceMatchScope") will be supported for filtering policies in 
a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for 
filtering policies.

The values supported for "resourceMatchScope" option are:

"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If 
resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, 
column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which 
matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"

If resourceMatchScope is specified, but its value is not one of "self", 
"ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".

An example curl command is as follows:

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0:path=/demo=self_or_ancestor'

This will return all access policies for cl1_hadoop service which match path 
'/demo' or any path that starts with '/demo/'

Similarly, a command

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0:udf=demo:database=tmp=self

will return only policies which have both database=tmp and udf=demo as one of 
their policy values.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
 3cdf40b 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
 fa2b940 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java
 8a784b4 
  
agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerPathResourceMatcher.java
 300c1f8 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
 36a9a27 
  agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
8f6426c 
  
agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java
 PRE-CREATION 
  
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_hdfs_policy.json
 PRE-CREATION 
  
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_policy.json
 PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
15f205a 
  security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 
4fb52a4 

Diff: https://reviews.apache.org/r/56640/diff/


Testing
---

Tested with local VM with and without specifying "resourceMatchScope" option in 
the filter-spec.


Thanks,

Abhay Kulkarni

Re: Review Request 56640: Support for using resource-matcher for filtering policies within a service if service-resource is provided in the filter

2017-02-23 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/#review166573
---


Fix it, then Ship it!





security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 
2138)


I think it will be helpful to document list of valid values 
(self/ancestor/self_or_ancestor) in SearchFilter.RESOURCE_MATCHER_SCOPE 
defintion.



security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 
2200)


foreach (matcher)
  foreach(policy)
   ... // this could endup adding the same policy multiple times, if it 
matches more than one matcher.

Consider rewritting this as:

foreach (policy)
  foreach(matcher)
   ...


- Madhan Neethiraj


On Feb. 22, 2017, 11:37 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56640/
> ---
> 
> (Updated Feb. 22, 2017, 11:37 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1383
> https://issues.apache.org/jira/browse/RANGER-1383
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger admin's REST API support retrieving and filtering policies for 
> resource specified in the provided filter. Currently, a simple string-match 
> and wildcard-match is used to filter policies. It is desirable to provide an 
> option to use, for filtering purpose, the same resource-matching algorithm 
> that is used by the policy engine to search policies that need to be 
> evaluated for access determination in the component.
> 
> A new option ("resourceMatchScope") will be supported for filtering policies 
> in a service. If it is required to filter policies based on
> the resources, then, with this option, Ranger will use resource-matchers for 
> filtering policies.
> 
> The values supported for "resourceMatchScope" option are:
> 
> "self" -> Search for exact match
> "ancestor" -> Search for policies which partially match specified resource. 
> If resource is incompletely specified (for example, if
> service-type supports multiple resourcedefs - hive supports database, table, 
> column; hbase supports database, column-family, column),
> then unspecified resourcedefs will be considered to have value of "*", which 
> matches any value.
> "self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
> 
> If resourceMatchScope is specified, but its value is not one of "self", 
> "ancestor" or "self_or_ancestor", then value is set to
> "self_or_ancestor".
> 
> An example curl command is as follows:
> 
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
> application/json" -X GET 
> 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0:path=/demo=self_or_ancestor'
> 
> This will return all access policies for cl1_hadoop service which match path 
> '/demo' or any path that starts with '/demo/'
> 
> Similarly, a command
> 
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
> application/json" -X GET 
> 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0:udf=demo:database=tmp=self
> 
> will return only policies which have both database=tmp and udf=demo as one of 
> their policy values.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
>  3cdf40b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
>  fa2b940 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java
>  8a784b4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerPathResourceMatcher.java
>  300c1f8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
>  36a9a27 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
> 8f6426c 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java
>  PRE-CREATION 
>   
> agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_hdfs_policy.json
>  PRE-CREATION 
>   
> agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_policy.json
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 15f205a 
>   security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 
> 4fb52a4 
> 
> Diff: https://reviews.apache.org/r/56640/diff/
> 
> 
> Testing
>

Re: Review Request 56640: Support for using resource-matcher for filtering policies within a service if service-resource is provided in the filter

2017-02-22 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
---

(Updated Feb. 22, 2017, 11:37 p.m.)


Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Changes
---

Addressed review comments.


Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383


Repository: ranger


Description
---

Ranger admin's REST API support retrieving and filtering policies for resource 
specified in the provided filter. Currently, a simple string-match and 
wildcard-match is used to filter policies. It is desirable to provide an option 
to use, for filtering purpose, the same resource-matching algorithm that is 
used by the policy engine to search policies that need to be evaluated for 
access determination in the component.

A new option ("resourceMatchScope") will be supported for filtering policies in 
a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for 
filtering policies.

The values supported for "resourceMatchScope" option are:

"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If 
resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, 
column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which 
matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"

If resourceMatchScope is specified, but its value is not one of "self", 
"ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".

An example curl command is as follows:

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0:path=/demo=self_or_ancestor'

This will return all access policies for cl1_hadoop service which match path 
'/demo' or any path that starts with '/demo/'

Similarly, a command

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0:udf=demo:database=tmp=self

will return only policies which have both database=tmp and udf=demo as one of 
their policy values.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
 3cdf40b 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
 fa2b940 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java
 8a784b4 
  
agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerPathResourceMatcher.java
 300c1f8 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
 36a9a27 
  agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
8f6426c 
  
agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java
 PRE-CREATION 
  
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_hdfs_policy.json
 PRE-CREATION 
  
agents-common/src/test/resources/resourcematcher/test_defaultpolicyresourcematcher_for_policy.json
 PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
15f205a 
  security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 
4fb52a4 

Diff: https://reviews.apache.org/r/56640/diff/


Testing
---

Tested with local VM with and without specifying "resourceMatchScope" option in 
the filter-spec.


Thanks,

Abhay Kulkarni

Re: Review Request 56640: Support for using resource-matcher for filtering policies within a service if service-resource is provided in the filter

2017-02-21 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
---

(Updated Feb. 21, 2017, 10:11 p.m.)


Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Changes
---

Addressed review comments


Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383


Repository: ranger


Description
---

Ranger admin's REST API support retrieving and filtering policies for resource 
specified in the provided filter. Currently, a simple string-match and 
wildcard-match is used to filter policies. It is desirable to provide an option 
to use, for filtering purpose, the same resource-matching algorithm that is 
used by the policy engine to search policies that need to be evaluated for 
access determination in the component.

A new option ("resourceMatchScope") will be supported for filtering policies in 
a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for 
filtering policies.

The values supported for "resourceMatchScope" option are:

"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If 
resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, 
column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which 
matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"

If resourceMatchScope is specified, but its value is not one of "self", 
"ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".

An example curl command is as follows:

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0:path=/demo=self_or_ancestor'

This will return all access policies for cl1_hadoop service which match path 
'/demo' or any path that starts with '/demo/'

Similarly, a command

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0:udf=demo:database=tmp=self

will return only policies which have both database=tmp and udf=demo as one of 
their policy values.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
 3cdf40b 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
 fa2b940 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java
 8a784b4 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
 36a9a27 
  agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
8f6426c 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
15f205a 
  security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 
4fb52a4 

Diff: https://reviews.apache.org/r/56640/diff/


Testing
---

Tested with local VM with and without specifying "resourceMatchScope" option in 
the filter-spec.


Thanks,

Abhay Kulkarni

Re: Review Request 56640: Support for using resource-matcher for filtering policies within a service if service-resource is provided in the filter

2017-02-19 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/#review166043
---




agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
 (line 51)


Consider renaming: "getServiceDefForNonrecursivePathResourceMatchers" ==> 
"cloneServiceDefForResourceFiltering"



agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
 (line 370)


Qhy should serviceDef be given as argument to this method? Shouldn't the 
serviceDef be set for the resource-matcher via init(). How is the serviceDef in 
this method parameter different from the one given in init()?

It will help to add examples/usecases for match being attempted here. 
Especially details of exits from the 'for' loop at line #370.. value of { 
matchType, ret } at each iteration and at exit.



security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 
2151)


Consider abstracting the details of updating resourceName in a method like: 
updateResourceForFilter(filterResources, serviceDef)



security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java (line 
2161)


Instead of hardcoding resource names ("path" and "queue") and delimiters 
(".", "/"), consider reading these from serviceDef.resources.


- Madhan Neethiraj


On Feb. 16, 2017, 10:45 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56640/
> ---
> 
> (Updated Feb. 16, 2017, 10:45 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1383
> https://issues.apache.org/jira/browse/RANGER-1383
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger admin's REST API support retrieving and filtering policies for 
> resource specified in the provided filter. Currently, a simple string-match 
> and wildcard-match is used to filter policies. It is desirable to provide an 
> option to use, for filtering purpose, the same resource-matching algorithm 
> that is used by the policy engine to search policies that need to be 
> evaluated for access determination in the component.
> 
> A new option ("resourceMatchScope") will be supported for filtering policies 
> in a service. If it is required to filter policies based on
> the resources, then, with this option, Ranger will use resource-matchers for 
> filtering policies.
> 
> The values supported for "resourceMatchScope" option are:
> 
> "self" -> Search for exact match
> "ancestor" -> Search for policies which partially match specified resource. 
> If resource is incompletely specified (for example, if
> service-type supports multiple resourcedefs - hive supports database, table, 
> column; hbase supports database, column-family, column),
> then unspecified resourcedefs will be considered to have value of "*", which 
> matches any value.
> "self_or_ancestor" -> Search for policies which match as "self" or "ancestor"
> 
> If resourceMatchScope is specified, but its value is not one of "self", 
> "ancestor" or "self_or_ancestor", then value is set to
> "self_or_ancestor".
> 
> An example curl command is as follows:
> 
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
> application/json" -X GET 
> 'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0:path=/demo=self_or_ancestor'
> 
> This will return all access policies for cl1_hadoop service which match path 
> '/demo' or any path that starts with '/demo/'
> 
> Similarly, a command
> 
> curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
> application/json" -X GET 
> 'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0:udf=demo:database=tmp=self
> 
> will return only policies which have both database=tmp and udf=demo as one of 
> their policy values.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
>  3cdf40b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
>  fa2b940 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java
>  8a784b4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
>  36a9a27 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
> 8f6426c 
>

Re: Review Request 56640: Support for using resource-matcher for filtering policies within a service if service-resource is provided in the filter

2017-02-15 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56640/
---

(Updated Feb. 16, 2017, 12:16 a.m.)


Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Changes
---

Reworked algorithm for matching policy against a RangerPolicyResourceMatcher


Bugs: RANGER-1383
https://issues.apache.org/jira/browse/RANGER-1383


Repository: ranger


Description
---

Ranger admin's REST API support retrieving and filtering policies for resource 
specified in the provided filter. Currently, a simple string-match and 
wildcard-match is used to filter policies. It is desirable to provide an option 
to use, for filtering purpose, the same resource-matching algorithm that is 
used by the policy engine to search policies that need to be evaluated for 
access determination in the component.

A new option ("resourceMatchScope") will be supported for filtering policies in 
a service. If it is required to filter policies based on
the resources, then, with this option, Ranger will use resource-matchers for 
filtering policies.

The values supported for "resourceMatchScope" option are:

"self" -> Search for exact match
"ancestor" -> Search for policies which partially match specified resource. If 
resource is incompletely specified (for example, if
service-type supports multiple resourcedefs - hive supports database, table, 
column; hbase supports database, column-family, column),
then unspecified resourcedefs will be considered to have value of "*", which 
matches any value.
"self_or_ancestor" -> Search for policies which match as "self" or "ancestor"

If resourceMatchScope is specified, but its value is not one of "self", 
"ancestor" or "self_or_ancestor", then value is set to
"self_or_ancestor".

An example curl command is as follows:

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hadoop?policyType=0:path=/demo=self_or_ancestor'

This will return all access policies for cl1_hadoop service which match path 
'/demo' or any path that starts with '/demo/'

Similarly, a command

curl -u admin:admin -H "Accept: application/json" -H "Content-Type: 
application/json" -X GET 
'http://localhost:6080/service/plugins/policies/service/name/cl1_hive?policyType=0:udf=demo:database=tmp=self

will return only policies which have both database=tmp and udf=demo as one of 
their policy values.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
 3cdf40b 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
 fa2b940 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerPolicyResourceMatcher.java
 8a784b4 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
 36a9a27 
  agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
8f6426c 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
15f205a 
  security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java 
4fb52a4 

Diff: https://reviews.apache.org/r/56640/diff/


Testing
---

Tested with local VM with and without specifying "resourceMatchScope" option in 
the filter-spec.


Thanks,

Abhay Kulkarni