Re: Review Request 74672: RANGER-4474: adventures in abac - part-2

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74672/#review225868
---


Ship it!




Ship It!

- Don Bosco Durai


On Oct. 16, 2023, 6:03 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74672/
> ---
> 
> (Updated Oct. 16, 2023, 6:03 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Barbara Eckman, Don Bosco Durai, 
> Abhay Kulkarni, Mehul Parikh, Monika Kachhadiya, Ramesh Mani, Selvamohan 
> Neethiraj, Subhrat Chaudhary, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4474
> https://issues.apache.org/jira/browse/RANGER-4474
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Blog #2 of the series exploring choices for setting up access control based 
> on attributes of the user and tags.
> 
> 
> Diffs
> -
> 
>   
> docs/src/site/resources/blogs/adventures_in_abac_2.files/fig1-policy_globalsalespartners_row_filter_sr_sp.jpg
>  PRE-CREATION 
>   
> docs/src/site/resources/blogs/adventures_in_abac_2.files/fig2-policy_globalsalespartners_row_filter_abac.jpg
>  PRE-CREATION 
>   
> docs/src/site/resources/blogs/adventures_in_abac_2.files/fig3-policy_tag_based_on_user_role.jpg
>  PRE-CREATION 
>   
> docs/src/site/resources/blogs/adventures_in_abac_2.files/fig4-policy_tag_policy_abac.jpg
>  PRE-CREATION 
>   
> docs/src/site/resources/blogs/adventures_in_abac_2.files/table_globalsalespartners.jpg
>  PRE-CREATION 
>   docs/src/site/resources/blogs/adventures_in_abac_2.html PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74672/diff/1/
> 
> 
> Testing
> ---
> 
> - Built docs directory contents using: mvn clean site:run
> - verified blogs list is rendered in localhost:8080/blogs.html
> - verified this blog is rendered in 
> http://localhost:8080/blogs/adventures_in_abac_2.html
> 
> 
> File Attachments
> 
> 
> PDF version of the blog
>   
> https://reviews.apache.org/media/uploaded/files/2023/10/16/adeaf837-7714-4e05-872a-ad018c833576__Adventures_in_ABAC_-_Part_2.pdf
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74609: RANGER-4234

[Don Bosco Durai]

> On Sept. 18, 2023, 6:18 p.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
> > Lines 51 (patched)
> > <https://reviews.apache.org/r/74609/diff/1/?file=2279790#file2279790line51>
> >
> > I hope this won't throw an out of bound exception, but only returns null
> 
> Barbara Eckman wrote:
> It only returns null.  Here's what happens for the input 'var result = 
> [[USER.dataCollection_Sensitivity]]. filter(function(n){ return [[n,"_"]][0] 
> == TAG.value}); result.length !=0', for the 2 iterations of the while loop:
> INFO [main] (JavaScriptEdits.java:56) - ==> 
> s0=[[USER.dataCollection_Sensitivity]] s1=USER.dataCollection_Sensitivity 
> s2=null delim=,
>  INFO [main] (JavaScriptEdits.java:56) - ==> s0=[[n,"_"]] s1=n s2=,"_" 
> delim=_

Thanks for the clarification


> On Sept. 18, 2023, 6:18 p.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
> > Lines 56 (patched)
> > <https://reviews.apache.org/r/74609/diff/1/?file=2279790#file2279790line56>
> >
> > Would there be an case where s0 to s2 will be null or not present
> 
> Barbara Eckman wrote:
> if m.find() is true, then s0 and s1 won't be null, because they represent 
> what was found.  s2 can be null, as we saw above, with no problems.

Thanks for the clarification


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/#review225767
---


On Sept. 18, 2023, 8:36 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74609/
> ---
> 
> (Updated Sept. 18, 2023, 8:36 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
> policy conditions
> 
> 
> Diffs
> -
> 
>   agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
>   agents-common/pom.xml b753c1368 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 9e5a94b1a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
>  7ac20764f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74609/diff/3/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/fe38eef9-22e6-4c91-85a0-46fe337ba87b__0001-doubleBracketsReplace-stuff.patch
> 0001-doubleBracketsReplace-stuff.patch
>   
> https://reviews.apache.org/media/uploaded/files/2023/09/18/062f1050-96d7-4ed0-9008-fd65311ea7b0__0001-doubleBracketsReplace-stuff.patch
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74609: RANGER-4234

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74609/#review225767
---




agents-common/pom.xml
Line 20 (original), 20 (patched)
<https://reviews.apache.org/r/74609/#comment314185>

Should we have the prefix "-3.0.0-20230918"?



agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
Lines 51 (patched)
<https://reviews.apache.org/r/74609/#comment314186>

I hope this won't throw an out of bound exception, but only returns null



agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
Lines 56 (patched)
<https://reviews.apache.org/r/74609/#comment314187>

Would there be an case where s0 to s2 will be null or not present



agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
Lines 60 (patched)
<https://reviews.apache.org/r/74609/#comment314188>

Should we store the original string and log both here to see what the 
original string was transformed?


- Don Bosco Durai


On Sept. 18, 2023, 4:48 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74609/
> ---
> 
> (Updated Sept. 18, 2023, 4:48 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4234: Eliminate need for splitting delimited strings into arrays in 
> policy conditions
> 
> 
> Diffs
> -
> 
>   agents-common/dev-support/spotbugsIncludeFile.xml PRE-CREATION 
>   agents-common/pom.xml b753c1368 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 9e5a94b1a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
>  7ac20764f 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/JavaScriptEdits.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74609/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74582: RANGER-4398: security-zone API enhancements to support incremental updates and resource pagination

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74582/#review225753
---


Ship it!




Ship It!

- Don Bosco Durai


On Sept. 8, 2023, 4:28 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74582/
> ---
> 
> (Updated Sept. 8, 2023, 4:28 p.m.)
> 
> 
> Review request for ranger, Anand Nadar, Abhay Kulkarni, Mehul Parikh, Monika 
> Kachhadiya, Pradeep Agrawal, Ramesh Mani, Subhrat Chaudhary, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-4398
> https://issues.apache.org/jira/browse/RANGER-4398
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> - updated RangerSecurityZone with additional of following fields for each 
> resource: id, createdBy/Time, updatedBy/Time
> - introduced RangerSecurityZoneV2, a wrapper over RangerSecurityZone, to 
> support partial updates
> - added following REST APIs and corresponding Python APIs:
> -- POST   /service/public/v2/api/zones-v2
> -- PUT/service/public/v2/api/zones-v2/{id}
> -- PUT/service/public/v2/api/zones-v2/{id}/partial
> -- GET/service/public/v2/api/zones-v2
> -- GET/service/public/v2/api/zones-v2/name/{name}
> -- GET/service/public/v2/api/zones-v2/{id}
> -- GET/service/public/v2/api/zones-v2/name/{name}/resources/{serviceName}
> -- GET/service/public/v2/api/zones-v2/{id}/resources/{serviceName}
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPrincipal.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java
>  71d64ca83 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZoneV2.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSecurityZoneHelper.java
>  PRE-CREATION 
>   intg/src/main/python/apache_ranger/client/ranger_client.py 484a42128 
>   intg/src/main/python/apache_ranger/model/ranger_base.py 2111534d0 
>   intg/src/main/python/apache_ranger/model/ranger_principal.py PRE-CREATION 
>   intg/src/main/python/apache_ranger/model/ranger_security_zone.py 6faa15744 
>   intg/src/main/python/setup.py 0a4b1c66e 
>   ranger-examples/sample-client/src/main/python/security_zone_v2.py 
> PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/rest/PublicAPIsv2.java 
> cd906ed22 
>   security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java 
> 55d6aaac5 
> 
> 
> Diff: https://reviews.apache.org/r/74582/diff/4/
> 
> 
> Testing
> ---
> 
> - verified that new REST APIs work correctly using Python scripts (included 
> in this patch)
> - verified that all existing tests pass successfully
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74517: RANGER-4320: createPrincipalsIfAbsent request parameter is not recognized by importPoliciesFromFile REST API

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74517/#review225609
---


Ship it!




Ship It!

- Don Bosco Durai


On July 15, 2023, 6:19 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74517/
> ---
> 
> (Updated July 15, 2023, 6:19 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Abhay Kulkarni, Ramesh Mani, Subhrat 
> Chaudhary, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4320
> https://issues.apache.org/jira/browse/RANGER-4320
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> updated REST APIs to use operation-context consistently so that REST APIs 
> that create/update policies will recognize createPrincipalsIfAbsent parameter
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
> 7a7cc8137 
>   security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java ca9f286b3 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> a307293eb 
>   
> security-admin/src/main/java/org/apache/ranger/security/context/RangerContextHolder.java
>  e42dc2406 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSecurityContextFormationFilter.java
>  fee1d5895 
> 
> 
> Diff: https://reviews.apache.org/r/74517/diff/1/
> 
> 
> Testing
> ---
> 
> - verified that importPoliciesFromFile REST API recognizes 
> createPrincipalsIfAbsent
> - all tests pass successfully
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: [VOTE] Apache Ranger 2.4.0 Release - rc2

[Don Bosco Durai]

+1

Selva, thank you for putting this together.

Bosco

On Mon, Mar 27, 2023 at 8:40 PM Selvamohan Neethiraj 
wrote:

> Rangers:
>
> Apache Ranger 2.4.0 release candidate #2 is now available for a vote
> within the dev community.
> Links to the release artifacts are given below. Please review and vote.
>
> The vote will be open for at least 72 hours or until necessary votes are
> reached.
> [   ] +1 approve
> [   ] +0 no opinion
> [   ] -1 disapprove (and reason why)
>
> Thanks,
> Selva-
> Ranger PMC
>
> List of issues / improvements addressed in this release:  click-here <
> https://issues.apache.org/jira/browse/RANGER-4154?jql=project=RANGER%20and%20fixVersion%20%20=%202.4.0%20and%20status%20=%20Resolved%20ORDER%20BY%20key%20desc
> >
>
> Git tag for the release:
> https://github.com/apache/ranger/tree/release-2.4.0-rc2
> Sources for the release:
> https://dist.apache.org/repos/dist/dev/ranger/2.4.0-rc2/apache-ranger-2.4.0.tar.gz
>
> Source release verification:
> PGP Signature:
> https://dist.apache.org/repos/dist/dev/ranger/2.4.0-rc2/apache-ranger-2.4.0.tar.gz.asc
> SHA256
> 
> Hash:
> https://dist.apache.org/repos/dist/dev/ranger/2.4.0-rc2/apache-ranger-2.4.0.tar.gz.sha256
> SHA512
> 
> Hash:
> https://dist.apache.org/repos/dist/dev/ranger/2.4.0-rc2/apache-ranger-2.4.0.tar.gz.sha512
>
> Keys to verify the signature:
> https://dist.apache.org/repos/dist/release/ranger/KEYS
>
> Click Here <
> https://issues.apache.org/jira/issues/?jql=project=RANGER%20and%20fixVersion%20%20=%202.4.0%20and%20status%20=%20Resolved%20and%20type%20in%20(%22New%20Feature%22,%20Improvement)%20ORDER%20BY%20key%20desc>
> to view New Features/Enhancements in this release.
>
>
>
>
>
>

Re: Review Request 74356: RANGER-4144 : Fixing test suite for kafka plugin

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74356/#review225291
---


Ship it!




Ship It!

- Don Bosco Durai


On March 20, 2023, 7:43 p.m., Selvamohan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74356/
> ---
> 
> (Updated March 20, 2023, 7:43 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Madhan Neethiraj, Ramesh Mani, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4144
> https://issues.apache.org/jira/browse/RANGER-4144
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Changed version of curator test version from 2.12.0 to 5.4.0 & fixed Java 
> Autoboxing issues
> 
> 
> Diffs
> -
> 
>   
> plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
>  d24ee1e57 
>   pom.xml 1e77f5191 
> 
> 
> Diff: https://reviews.apache.org/r/74356/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Run Unit Test (as part of build) done on Linux Platform
> 
> 
> Thanks,
> 
> Selvamohan Neethiraj
> 
>

Re: Review Request 74356: RANGER-4144 : Fixing test suite for kafka plugin

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74356/#review225290
---



Approved


plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
Lines 110 (patched)
<https://reviews.apache.org/r/74356/#comment313984>

Seems IOException will come only from line 105, this review comment is not 
valid. I am approving the changes


- Don Bosco Durai


On March 20, 2023, 7:43 p.m., Selvamohan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74356/
> ---
> 
> (Updated March 20, 2023, 7:43 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Madhan Neethiraj, Ramesh Mani, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4144
> https://issues.apache.org/jira/browse/RANGER-4144
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Changed version of curator test version from 2.12.0 to 5.4.0 & fixed Java 
> Autoboxing issues
> 
> 
> Diffs
> -
> 
>   
> plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
>  d24ee1e57 
>   pom.xml 1e77f5191 
> 
> 
> Diff: https://reviews.apache.org/r/74356/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Run Unit Test (as part of build) done on Linux Platform
> 
> 
> Thanks,
> 
> Selvamohan Neethiraj
> 
>

Re: Review Request 74356: RANGER-4144 : Fixing test suite for kafka plugin

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74356/#review225288
---




plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
Lines 110 (patched)
<https://reviews.apache.org/r/74356/#comment313982>

Is this message misleading? Even line 106 will through exception. Would it 
be the same reason for port not available?


- Don Bosco Durai


On March 20, 2023, 7:43 p.m., Selvamohan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74356/
> ---
> 
> (Updated March 20, 2023, 7:43 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Madhan Neethiraj, Ramesh Mani, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4144
> https://issues.apache.org/jira/browse/RANGER-4144
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Changed version of curator test version from 2.12.0 to 5.4.0 & fixed Java 
> Autoboxing issues
> 
> 
> Diffs
> -
> 
>   
> plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
>  d24ee1e57 
>   pom.xml 1e77f5191 
> 
> 
> Diff: https://reviews.apache.org/r/74356/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Run Unit Test (as part of build) done on Linux Platform
> 
> 
> Thanks,
> 
> Selvamohan Neethiraj
> 
>

Re: Review Request 74253: This script will help anyone to setup ranger within few minutes of downloading Apache Ranger Repo.

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74253/#review225015
---


Ship it!




Ship It!

- Don Bosco Durai


On Dec. 21, 2022, 2:50 a.m., Selvamohan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74253/
> ---
> 
> (Updated Dec. 21, 2022, 2:50 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4017
> https://issues.apache.org/jira/browse/RANGER-4017
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-4017: Able to run Apache Ranger after downloading the source from REPO 
> by executing a single command (& within few minutes)
> 
> 
> Diffs
> -
> 
>   README.txt fce972ab1b986e7f1d28cf4e35f086929cf9169a 
>   dev-support/ranger-docker/Dockerfile.ranger-base 
> a4bb9008ff6e0d16784e286d22ab353e26be811c 
>   dev-support/ranger-docker/docker-compose.ranger-build.yml 
> f0b5b05e0cb696722297a83b7a507dc954f43398 
>   ranger_in_docker PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/74253/diff/4/
> 
> 
> Testing
> ---
> 
> Tested the script in Mac (M2) and Linux (Ubundu) 
> Documented the execution steps in README.txt
> 
> 
> Thanks,
> 
> Selvamohan Neethiraj
> 
>

Re: Review Request 74216: RANGER-3982: updated Python client to support Ranger KMS REST APIs

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74216/#review224927
---


Ship it!




Ship It!

- Don Bosco Durai


On Nov. 24, 2022, 8:28 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74216/
> ---
> 
> (Updated Nov. 24, 2022, 8:28 p.m.)
> 
> 
> Review request for ranger, Abhishek  Kumar, Ankita Sinha, deepak sharma, 
> Dhaval Shah, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Monika 
> Kachhadiya, Ramesh Mani, Siddhesh Phatak, Sailaja Polavarapu, Subhrat 
> Chaudhary, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3982
> https://issues.apache.org/jira/browse/RANGER-3982
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> - added classes RangerKMSClient, RangerKey, RangerKeyVersion, 
> RangerKeyMetadata and RangerEncryptedKeyVersion
> - updated README.md with sample calls for each KMS API
> 
> 
> Diffs
> -
> 
>   intg/src/main/python/README.md f21628fb0 
>   intg/src/main/python/apache_ranger/client/ranger_client.py 7bb6493a1 
>   intg/src/main/python/apache_ranger/client/ranger_kms_client.py PRE-CREATION 
>   intg/src/main/python/apache_ranger/exceptions.py a2299479c 
>   intg/src/main/python/apache_ranger/model/ranger_base.py 83ec44dd4 
>   intg/src/main/python/apache_ranger/model/ranger_kms.py PRE-CREATION 
>   intg/src/main/python/apache_ranger/utils.py b0ceb5c59 
>   intg/src/main/python/setup.py 8ea476320 
> 
> 
> Diff: https://reviews.apache.org/r/74216/diff/2/
> 
> 
> Testing
> ---
> 
> - verified KMS API calls using test_ranger_kms.py (in README.md)
> - test Apache Ranger Python client (0.0.19) that includes this patch is 
> available at https://test.pypi.org/project/apache-ranger/
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74178: RANGER-3953: fix potential NPE in policy-engine initialization

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74178/#review224819
---




agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
Lines 122 (patched)
<https://reviews.apache.org/r/74178/#comment313670>

Since the parameter "script" is user entered, should we catch the exception 
and log/ignore it?


- Don Bosco Durai


On Oct. 20, 2022, 3:09 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74178/
> ---
> 
> (Updated Oct. 20, 2022, 3:09 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Kishor Gollapalliwar, Abhay 
> Kulkarni, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, 
> Subhrat Chaudhary, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3953
> https://issues.apache.org/jira/browse/RANGER-3953
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> updated to handle null values gracefully
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerRequestScriptEvaluator.java
>  6ad2a144e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/resourcematcher/RangerURLResourceMatcher.java
>  1a6b52f8c 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/MacroProcessor.java 
> 77091332c 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRequestExprResolver.java
>  3a183cff6 
>   security-admin/src/main/java/org/apache/ranger/common/StringUtil.java 
> ed2e8df77 
> 
> 
> Diff: https://reviews.apache.org/r/74178/diff/1/
> 
> 
> Testing
> ---
> 
> - verified that row-filter policies with empty filter expression don't result 
> in NPE
> - verified that all exist tests pass successfully
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 74170: RANGER-3947 fix thread leak in SolrCollectionBootstrapper

[Don Bosco Durai]

> On Oct. 15, 2022, 4:42 p.m., Don Bosco Durai wrote:
> > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
> > Lines 204 (patched)
> > <https://reviews.apache.org/r/74170/diff/1/?file=2270661#file2270661line204>
> >
> > Should set solrCloudClient to null after close? Also what happens if we 
> > set to null, would it get created again?
> 
> Sai Sandeep Rangisetti wrote:
> After setting it to null it gets created in the connect method. So I 
> think we don't need to set it to null as it always gets created in the loop

Thanks for clarifying this.


> On Oct. 15, 2022, 4:42 p.m., Don Bosco Durai wrote:
> > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
> > Lines 207 (patched)
> > <https://reviews.apache.org/r/74170/diff/1/?file=2270661#file2270661line207>
> >
> > 1. Any reason we are setting this as severe? Can we use the level WARN 
> > or ERROR?
> > 2. Can we pass the exception as parameter to the logger?
> 
> Sai Sandeep Rangisetti wrote:
> 1. This class is using `java.util.logging.Logger` which does not have the 
> level ERROR. It only has severe and warning
> 2. java.util.logging.Logger does not have method to pass exception as 
> parameter in the `logger.warning` or `logger.severe` methods. It has 
> `logger.log(Level, Exception, Supplier)` method but I didn't see it being 
> used in this class so I also didn't use it

Thanks for clarifying this.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74170/#review224798
---


On Oct. 14, 2022, 3:07 p.m., Sai Sandeep Rangisetti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74170/
> ---
> 
> (Updated Oct. 14, 2022, 3:07 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-3947
> https://issues.apache.org/jira/browse/RANGER-3947
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Closing the solr cloud client in SolrCollectionBootstrapper's retry loop of 
> creating solr config and collection. Without this new solr cloud client is 
> created in every loop and new connection pools which will not be cleaned up 
> and create large number of threads
> 
> 
> Diffs
> -
> 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
>  fe4006f76 
> 
> 
> Diff: https://reviews.apache.org/r/74170/diff/1/
> 
> 
> Testing
> ---
> 
> Ran ranger-admin without ranger_audit config in zk and no 
> contrib/solr_for_audit_setup/conf file which leads to retry loop and verified 
> that threads aren't increasinng
> 
> 
> Thanks,
> 
> Sai Sandeep Rangisetti
> 
>

Re: Review Request 74170: RANGER-3947 fix thread leak in SolrCollectionBootstrapper

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74170/#review224798
---




embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
Lines 204 (patched)
<https://reviews.apache.org/r/74170/#comment313629>

Should set solrCloudClient to null after close? Also what happens if we set 
to null, would it get created again?



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
Lines 207 (patched)
<https://reviews.apache.org/r/74170/#comment313630>

1. Any reason we are setting this as severe? Can we use the level WARN or 
ERROR?
2. Can we pass the exception as parameter to the logger?


- Don Bosco Durai


On Oct. 14, 2022, 3:07 p.m., Sai Sandeep Rangisetti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74170/
> ---
> 
> (Updated Oct. 14, 2022, 3:07 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-3947
> https://issues.apache.org/jira/browse/RANGER-3947
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Closing the solr cloud client in SolrCollectionBootstrapper's retry loop of 
> creating solr config and collection. Without this new solr cloud client is 
> created in every loop and new connection pools which will not be cleaned up 
> and create large number of threads
> 
> 
> Diffs
> -
> 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
>  fe4006f76 
> 
> 
> Diff: https://reviews.apache.org/r/74170/diff/1/
> 
> 
> Testing
> ---
> 
> Ran ranger-admin without ranger_audit config in zk and no 
> contrib/solr_for_audit_setup/conf file which leads to retry loop and verified 
> that threads aren't increasinng
> 
> 
> Thanks,
> 
> Sai Sandeep Rangisetti
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74142/#review224797
---




agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
Lines 44 (patched)
<https://reviews.apache.org/r/74142/#comment313628>

Yes, it was a question. I was not sure whether we are printing would have 
sensitive information. If it doesn't the suggestion is not to print them. The 
reason being, it is common for applications to be configured to DEBUG level 
during troubleshooting sessions and also in some cases, these logs are sent to 
external systems like DataDog (in the cloud) or other log aggregation tools and 
it would be difficult to enforce any policies in those tools.


- Don Bosco Durai


On Sept. 26, 2022, 7:17 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Sept. 26, 2022, 7:17 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRoleUserStoreRetriever class, which transforms role membership into 
> user attributes, for ease of use in complex policy conditions.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetBearerToken.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromDataFile.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/GetFromURL.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/LICENSE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/NOTICE
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/README.md
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/TokenInputs.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/pom.xml
>  PRE-CREATION 
>   plugin-nestedstructure/README.md ea878f6a2 
> 
> 
> Diff: https://reviews.apache.org/r/74142/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Barbara Eckman
> 
>

Re: Review Request 74142: RangerExternalUserStoreRetriever class Ranger-3855

[Don Bosco Durai]

/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
Lines 55 (patched)
<https://reviews.apache.org/r/74142/#comment313555>

What is the purpose for this method? Is it just to read the entire file 
into a string object? If so, should we use class method like File.readString()?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
Lines 56 (patched)
<https://reviews.apache.org/r/74142/#comment313558>

To optimize on memory, should we StringBuffer here?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
Lines 59 (patched)
<https://reviews.apache.org/r/74142/#comment313557>

Can we use closable (try()) here? So that even there is an exception, the 
stream will be closed



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
Lines 66 (patched)
<https://reviews.apache.org/r/74142/#comment313559>

Should pass the exception as parameter? So we can get the stack trace?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
Lines 67 (patched)
<https://reviews.apache.org/r/74142/#comment313560>

Do we need to print this in stderr?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
Lines 73 (patched)
<https://reviews.apache.org/r/74142/#comment313561>

Can we use closable here? So that even on exception the stream is closed.



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/HandleSecrets.java
Lines 77 (patched)
<https://reviews.apache.org/r/74142/#comment313562>

Any reason we are supressing this error? Should we propagate for the caller 
so it can be handled appropriately?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
Lines 54 (patched)
<https://reviews.apache.org/r/74142/#comment313563>

Do we need to handle failure to getFromURL() method ?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
Lines 56 (patched)
<https://reviews.apache.org/r/74142/#comment313564>

Is it okay to ignore this exception?
Can we also remove the next line?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerExternalUserStoreRetriever.java
Lines 69 (patched)
<https://reviews.apache.org/r/74142/#comment313565>

This seems to be internal comcast class. What happens if this is not 
available in the opensource?



agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/externalUserStoreRetrievers/RangerRoleUserStoreRetriever.java
Lines 76 (patched)
<https://reviews.apache.org/r/74142/#comment313566>

Does need to be in the seperate line?


- Don Bosco Durai


On Sept. 26, 2022, 7:17 p.m., Barbara Eckman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74142/
> ---
> 
> (Updated Sept. 26, 2022, 7:17 p.m.)
> 
> 
> Review request for ranger and madhan.
> 
> 
> Bugs: Ranger-3855
> https://issues.apache.org/jira/browse/Ranger-3855
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerExternalUserStoreRetriever class Ranger-3855
> 
> Ranger version 3.0.0 provides a means, via a context enricher, to add or 
> retrieve attributes to the database of users for whom Ranger controls access. 
> This permits syntax like "Dumbo" in $USER.aliases any Ranger policy 
> condition, including row and tag filters.   This greatly enhances the ability 
> to provide custom Attribute-based Access Control based on the specific 
> business needs of one's organization.
> 
> I believe that the original assumption was that such attributes would be 
> added to AD/LDAP and enter Ranger via regular user sync's. However, this 
> process does not currently work with Azure AD, which many organizations use. 
> Neither does it provide timely support for organizations for whom adding each 
> new attribute to AD would be subject to prolonged scrutiny by overworked 
> security teams.  
> 
> In the spirit of the RangerAdminUserStoreRetriever context enricher, we have 
> written a RangerExternalUserStoreRetriever class which adds arbitrary 
> attributes to Ranger users via external API calls, thus freeing additions to 
> the UserStore from dependency on AD/LDAP.   We have also written a 
> RangerRol

Re: [ANNOUNCE] Apache Ranger 2.3.0 released

[Don Bosco Durai]

This is great. Thanks everyone.

 

Regards

 

Bosco

 

 

From: Madhan Neethiraj 
Reply-To: 
Date: Sunday, July 10, 2022 at 10:13 PM
To: , 
Subject: Re: [ANNOUNCE] Apache Ranger 2.3.0 released

 

Ramesh - thank you for driving this release.

 

Rangers - thank you all for your contributions to this release!

 

This release includes several important improvements:

  - Ranger KMS integration with Google cloud HSM, Tencent KMS

  - added support for Amazon CloudWatch as audit store

  - ability to scope delegated-admin to specific permissions

  - ability to use macros in conditions, like:

- IS_IN_GROUP('hr') && IS_IN_GROUP('finance')

- TAG.piiType == 'email'

  - attribute-based access control (ABAC) enhancements, with ability to refer 
user/group/tag attributes

   - resource names, like: /dept/${{USER.dept}},  db_${{USER._name}}

- row-filters, like country = ${{USER.country}}, store_id in 
(${{GET_UG_ATTR _CSV('managesStore')}})

- conditions, like: HAS_UG_ATTR('managesStore')

  - removal of log4j-1 dependency

  - performance improvements in multiple areas

  - improvements in Docker setup

 

Madhan

 

On 7/9/22, 8:42 PM, "Ramesh Mani"  wrote:

 

Dear all,

 

Apache Ranger team is happy to announce the release of Apache Ranger 2.3.0.

Apache Ranger is a framework to enable, monitor and manage comprehensive

data security across the Hadoop platform and beyond. Apache Ranger 2.3.0

contains a number of new features, improvements and bug fixes. Details can

be found in the release notes at


https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+2.3.0+-+Release+Notes

 

The release artifacts are available at:

https://ranger.apache.org/download.html The binary artifacts are available

from Maven central and its mirrors. In the initial 48 hours, the release

may not be available on all mirrors. When downloading from a mirror site,

please remember to verify the downloads using signatures found at:

https://www.apache.org/dist/ranger/KEYS More details on Apache Ranger can

be found at: https://ranger.apache.org We thank everyone who made this

release possible. Thanks, Apache Ranger team

Re: Need community feedback on expanding scope of Apache Ranger

[Don Bosco Durai]

Selva, thanks for starting the thread. 

  3.  What additional data sources/services would you like to have 
Apache Ranger authorization?
It would be good if we can start supporting Cloud data sources as part of the 
community. We should also look into other services like OPA and either leverage 
that or extend that.

  4.  What new features/enhancements would you like to see in 
Apache Ranger?
Few things like simplified deployment of Ranger, access approval workflow and 
synchronizing policies to Cloud data sources will be nice to have. And we need 
revamp of web pages ( 

Thanks

Bosco

On 6/30/22, 1:14 PM, "Madhan Neethiraj"  wrote:

(added u...@ranger.apache.org back in this thread)

cdmikechen - thank you for your response. 

Selva - thank you for starting this thread. Responses from the community 
will help prioritize next set of enhancements in Apache Ranger. Looking forward 
to more responses from Apache Ranger community.

I think enabling Apache Ranger authorization to services that don't support 
pluggable authorization will be of immense value. This is critical especially 
as enterprises use various cloud services where the service provider may not 
support a pluggable authorizer model. Such enhancements to authorization 
framework will help extend Apache Ranger authorization to services like 
Databricks SQL, Snowflake, Google BigQuery, AWS Redshift, MS-SQL, Postgres, and 
more.

Thanks,
Madhan

On 6/17/22, 12:09 AM, "陈 翔"  wrote:

1. My company has used ranger from 1.2.0 and updated to 2.3.0 
recently.

2. At present, my company mainly use ranger with hive and atlas.

3. I personally hope to support Drill or Druid.

4. I  hope that the installation of ranger can be easier (for 
example, remove Python dependency when installing), and I also hope that other 
authentication methods such as oauth2/oidc can be supported. At the same time, 
my company runs the Ranger service in k8s. We have developed an k8s operator to 
simplify deployment and using. If possible in the future, I will put relevant 
projects to Github.

5. Not yet.

Hopefully this will help the community~
Thanks


发件人: sneet...@apache.org 
日期: 星期五, 2022年6月17日 上午10:43
收件人: dev@ranger.apache.org , 
u...@ranger.apache.org , priv...@ranger.apache.org 

主题: Need community feedback on expanding scope of Apache Ranger
As Apache Ranger has taken a critical role in Security & Governance 
of the big data/Hadoop technologies, I would like to see if the scope of Apache 
Ranger can be extended to cover other areas such as CLOUD data sources. To 
identify the industry needs, can you please provide some feedback on your 
current Apache Ranger use and your future needs:


  1.  Do you currently use Apache Ranger? If so, What Version?
  2.  Apache Ranger supports authorization for 20+ data 
sources/services. For which data sources/services do you use Apache Ranger 
authorization?
  3.  What additional data sources/services would you like to have 
Apache Ranger authorization?
  4.  What new features/enhancements would you like to see in 
Apache Ranger?
  5.  Have you developed Apache Ranger plugins for any data 
sources/services? If yes, please list them.

As a community, we can review this (feedback) information to create 
more backlog tasks for us to scope the next major release.

Thanks,
Selva-

Re: Review Request 73106: RANGER-3132: mapping problem for AuthzAuditEvent between ActionType and Action

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73106/#review222384
---



Seems good. Thanks

- Don Bosco Durai


On Dec. 24, 2020, 3:51 p.m., Rodolphe Dugé de Bernonville wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73106/
> ---
> 
> (Updated Dec. 24, 2020, 3:51 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-3132: mapping problem for AuthzAuditEvent between ActionType and Action
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java
>  427372643 
> 
> 
> Diff: https://reviews.apache.org/r/73106/diff/1/
> 
> 
> Testing
> ---
> 
> agents-common, hbase-agent, hdfs-agent, hive-agent, plugin-atlas, 
> plugin-kafka, plugin-yarn compilation
> is it enough ?
> 
> 
> Thanks,
> 
> Rodolphe Dugé de Bernonville
> 
>

Re: [ANNOUNCE] Apache Ranger - Python client

[Don Bosco Durai]

Madhan, this is very useful.

Thanks again

Regards

Bosco

On 12/4/20, 11:35 PM, "Madhan Neethiraj"  wrote:

Bosco,

Yes. Python APIs support  CRUD operations on 
service-defs/services/policies/security-zones/roles - like:
  - create_policy()
  - update_policy()
  - update_policy_by_id()
  - apply_policy()
  - delete_policy()
  - delete_policy_by_id()
  - get_policy()
  - get_policy_by_id()
  - get_policies_in_service()
  - find_policies()

Until documentation is in place, complete list of APIs can be found from 
RangerClient class here:  
https://github.com/apache/ranger/blob/master/intg/src/main/python/apache_ranger/client/ranger_client.py#L245.
 

Hope this helps.

Madhan


On 12/4/20, 10:48 PM, "Don Bosco Durai"  wrote:

Madhan, this is very good.

Does the script also supports deleting or modifying of Ranger Policies?

Regards

Bosco


On 12/4/20, 9:09 AM, "Madhan Neethiraj"  wrote:

All,

Official Python client for Apache Ranger is now available at 
https://pypi.org/project/apache-ranger/. Python client APIs mirror Apache 
Ranger REST APIs, and enable administration of Apache Ranger using Python.

Here is a sample usage to create a service and a policy using 
Python client:

from apache_ranger.model.ranger_service import RangerService
from apache_ranger.client.ranger_client import RangerClient
from apache_ranger.model.ranger_policy  import RangerPolicy, 
RangerPolicyResource, RangerPolicyItem, RangerPolicyItemAccess

service_name = 'dev_hive'

service = RangerService(name=service_name, type='hive')
service.configs = {'username':'hive', 'password':'hive', 
'jdbc.driverClassName': 'org.apache.hive.jdbc.HiveDriver', 'jdbc.url': 
'jdfb:hive2://ranger-hadoop:1', 'hadoop.security.authorization': 'true'}

policy = RangerPolicy(service=service_name, name='test policy')
policy.resources = {'database': 
RangerPolicyResource(['test_db']), 'table': RangerPolicyResource(['test_tbl']), 
'column': RangerPolicyResource(['*'])}
policy.policyItems.append(RangerPolicyItem(users=['admin'], 
accesses=[RangerPolicyItemAccess('create'), RangerPolicyItemAccess('alter'), 
RangerPolicyItemAccess('drop')], delegateAdmin=True))
policy.denyPolicyItems.append(RangerPolicyItem(users=['admin'], 
accesses=[RangerPolicyItemAccess('select')]))


ranger_client   = RangerClient('http://localhost:6080', 
'admin', 'rangerR0cks!')
created_service = ranger_client.create_service(service)
created_policy  = ranger_client.create_policy(policy)

Apache Ranger team is updating the documentation to include details 
of Python APIs.

Your feedback and suggestions are welcome.

Thanks,
Madhan

Re: [ANNOUNCE] Apache Ranger - Python client

[Don Bosco Durai]

Madhan, this is very good.

Does the script also supports deleting or modifying of Ranger Policies?

Regards

Bosco


On 12/4/20, 9:09 AM, "Madhan Neethiraj"  wrote:

All,

Official Python client for Apache Ranger is now available at 
https://pypi.org/project/apache-ranger/. Python client APIs mirror Apache 
Ranger REST APIs, and enable administration of Apache Ranger using Python.

Here is a sample usage to create a service and a policy using Python client:

from apache_ranger.model.ranger_service import RangerService
from apache_ranger.client.ranger_client import RangerClient
from apache_ranger.model.ranger_policy  import RangerPolicy, 
RangerPolicyResource, RangerPolicyItem, RangerPolicyItemAccess

service_name = 'dev_hive'

service = RangerService(name=service_name, type='hive')
service.configs = {'username':'hive', 'password':'hive', 
'jdbc.driverClassName': 'org.apache.hive.jdbc.HiveDriver', 'jdbc.url': 
'jdfb:hive2://ranger-hadoop:1', 'hadoop.security.authorization': 'true'}

policy = RangerPolicy(service=service_name, name='test policy')
policy.resources = {'database': RangerPolicyResource(['test_db']), 
'table': RangerPolicyResource(['test_tbl']), 'column': 
RangerPolicyResource(['*'])}
policy.policyItems.append(RangerPolicyItem(users=['admin'], 
accesses=[RangerPolicyItemAccess('create'), RangerPolicyItemAccess('alter'), 
RangerPolicyItemAccess('drop')], delegateAdmin=True))
policy.denyPolicyItems.append(RangerPolicyItem(users=['admin'], 
accesses=[RangerPolicyItemAccess('select')]))


ranger_client   = RangerClient('http://localhost:6080', 'admin', 
'rangerR0cks!')
created_service = ranger_client.create_service(service)
created_policy  = ranger_client.create_policy(policy)

Apache Ranger team is updating the documentation to include details of 
Python APIs.

Your feedback and suggestions are welcome.

Thanks,
Madhan

Re: Review Request 72969: RANGER-3000:Audit-filter feature implementation to help reduce volume of audit logs generated

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72969/#review47
---




agents-common/src/main/java/org/apache/ranger/authorization/utils/JsonUtils.java
Lines 128 (patched)
<https://reviews.apache.org/r/72969/#comment311303>

We are returning null. I assume we are handling it where it is been called. 
Should we just throw the exception and let it get handle where it is called?



agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
Lines 907 (patched)
<https://reviews.apache.org/r/72969/#comment311304>

Should this be accessTime and not accessType?



agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerAuditPolicyEvaluator.java
Lines 74 (patched)
<https://reviews.apache.org/r/72969/#comment311305>

Let's make sure ++ doesn't mess the passed value. It might be better to 
increment in a new statement to be safe


- Don Bosco Durai


On Nov. 30, 2020, 3:42 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72969/
> ---
> 
> (Updated Nov. 30, 2020, 3:42 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-3000
> https://issues.apache.org/jira/browse/RANGER-3000
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-3000:Audit-filter feature implementation to help reduce volume of 
> audit logs generated
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/utils/JsonUtils.java
>  994d3944d 
>   agents-common/src/main/java/org/apache/ranger/plugin/model/AuditFilter.java 
> PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 04c6e75c4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerServiceDefHelper.java
>  0d9a346d4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessResult.java
>  c54ef1704 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngine.java
>  4e41adcea 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  b66d5a1ce 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
>  169ed0f5d 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerAuditPolicyEvaluator.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java
>  07fb63872 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
>  979488181 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  2d9bc7382 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  6ab068f6f 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyEngine.java
>  63fccd0b3 
>   
> agents-common/src/test/resources/policyengine/test_policyengine_audit_filter_hdfs.json
>  PRE-CREATION 
>   
> agents-common/src/test/resources/policyengine/test_policyengine_audit_filter_hive.json
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> 520258715 
> 
> 
> Diff: https://reviews.apache.org/r/72969/diff/5/
> 
> 
> Testing
> ---
> 
> - Testing done in local VM.
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: [VOTE] Apache Ranger 2.1.0 rc0

[Don Bosco Durai]

+1

- Build successfully 

Amazing work everyone. Madhan, thanks for driving this

Regards

Bosco


On 9/1/20, 12:25 PM, "Selvamohan Neethiraj"  wrote:

+1

Thanks Madhan for driving the release of Apache Ranger 2.1.0
* Downloaded source and build successfully.

Thanks,
Selva-

On 8/30/20 5:37 PM, Madhan Neethiraj wrote:
> Rangers,
> 
> Apache Ranger 2.1.0 release candidate #0 is now available for a vote 
within dev community. Links to the release artifacts are given below. Please 
review and vote.
> 
> The vote will be open for at least 72 hours or until necessary votes are 
reached.
>   [ ] +1 approve
>   [ ] +0 no opinion
>   [ ] -1 disapprove (and reason why)
> 
> Regards,
> Madhan
> 
> List of all issues addressed in this release: 
https://issues.apache.org/jira/issues/?jql=project=RANGER AND status=Resolved 
AND fixVersion=2.1.0 ORDER BY key DESC
> 
> Git tag for the release: 
https://github.com/apache/ranger/tree/release-2.1.0-rc0
> Sources for the release: 
https://dist.apache.org/repos/dist/dev/ranger/2.1.0-rc0/apache-ranger-2.1.0.tar.gz
> 
> Source release verification:
>   PGP Signature: 
https://dist.apache.org/repos/dist/dev/ranger/2.1.0-rc0/apache-ranger-2.1.0.tar.gz.asc
>   SHA256 Hash:   
https://dist.apache.org/repos/dist/dev/ranger/2.1.0-rc0/apache-ranger-2.1.0.tar.gz.sha256
>   SHA512 Hash:   
https://dist.apache.org/repos/dist/dev/ranger/2.1.0-rc0/apache-ranger-2.1.0.tar.gz.sha512
> 
> Keys to verify the signature of the release artifacts are available at: 
https://dist.apache.org/repos/dist/release/ranger/KEYS
> 
> New features/enhancements:
>   - Hive plugin enhancement to authorize based on database/table owners
>   - Solr plugin enhancement to support document level authorization
>   - Kafka plugin enhancement to support authorization on consumer-groups
>   - Presto plugin enhancements to support row-filtering and column-masking
>   - Atlas plugin enhancements to support authorization for new operations 
and resources
>   - Plugins enhancement to support Ranger HA without requiring a 
load-balancer
>   - Plugins enhancements to support incremental tag updates
>   - Plugins enhancements to support super-users and super-groups
>   - Plugins enhancements to support audit excluded-users
>   - Added support for Elastic Search as audit store
>   - Ranger Admin UI improvements
>   - Performance improvement in bulk create/update of policies
>   - Ranger KMS enhancement to support Azure Key Vault
>   - Java client library to access Ranger REST APIs
>   - Python client library to access Ranger REST APIs
>   - Added docker setup to build, deploy Apache Ranger along with Ranger 
authorization enabled HDFS/YARN/HBase/Kafka services
>   - updated versions of dependent libraries/components
> 
> 
> 

Re: Review Request 71921: Add support for ElasticSearch as an Audit Database

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71921/#review219835
---




agents-audit/pom.xml
Lines 118 (patched)
<https://reviews.apache.org/r/71921/#comment308058>

Any reason the version is hard coded?



agents-audit/src/main/java/org/apache/ranger/audit/destination/ElasticSearchAuditDestination.java
Lines 120 (patched)
<https://reviews.apache.org/r/71921/#comment308060>

Can we have this in if(LOG.isDebugEnabled()) { block }?



agents-audit/src/main/java/org/apache/ranger/audit/destination/ElasticSearchAuditDestination.java
Lines 160 (patched)
<https://reviews.apache.org/r/71921/#comment308061>

For debugging purpose, should we set this as member attributes and init 
load them and have it logged at INFO level (without the password)? This will 
help during debuging.



agents-audit/src/main/java/org/apache/ranger/audit/destination/ElasticSearchAuditDestination.java
Lines 200 (patched)
<https://reviews.apache.org/r/71921/#comment308062>

Can we print additional information like port, etc.



migration-util/ambari2.1-hdp2.3-ranger0.50/bin/import_ranger_to_ambari.py
Lines 292 (patched)
<https://reviews.apache.org/r/71921/#comment308063>

Do we need to update this file? Not sure whether anyone will need it. If it 
not used or tested, then we should probably not just add and risk breaking 
something else.



pom.xml
Lines 135 (patched)
<https://reviews.apache.org/r/71921/#comment308057>

Can we indent this as others?



security-admin/scripts/install.properties
Lines 85 (patched)
<https://reviews.apache.org/r/71921/#comment308064>

Can we keep plugin and admin config similar? In Plugin we are taking Port 
as config



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsService.java
Lines 76 (patched)
<https://reviews.apache.org/r/71921/#comment308065>

Just curious if there are additional information that can be printed which 
would be useful for debug?



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsService.java
Lines 105 (patched)
<https://reviews.apache.org/r/71921/#comment308066>

Can we print the request data? And also the execption?



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchMgr.java
Lines 58 (patched)
<https://reviews.apache.org/r/71921/#comment308067>

Seems we have the port configurable. But it is not in the properties file. 
It would be good to have it with the default value.



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchMgr.java
Lines 60 (patched)
<https://reviews.apache.org/r/71921/#comment308068>

Any reason this is WARN? Can we have it as INFO?



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchMgr.java
Lines 64 (patched)
<https://reviews.apache.org/r/71921/#comment308069>

I think we have org.apache.ranger.common.StringUtil.isEmpty() method which 
will check for null and empty string. It will take care if the value is ""



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchMgr.java
Lines 73 (patched)
<https://reviews.apache.org/r/71921/#comment308070>

Can we put an INFO log with the username that will be used to login? So we 
know it is trying authentication mechanism?



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchMgr.java
Lines 87 (patched)
<https://reviews.apache.org/r/71921/#comment308071>

Can we have other properties also?



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchMgr.java
Lines 99 (patched)
<https://reviews.apache.org/r/71921/#comment308072>

Since connect() is already synchronized, do we need this here also?



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchUtil.java
Lines 73 (patched)
<https://reviews.apache.org/r/71921/#comment308074>

Can we use StringUtil.isEmpty() here and remove empty string check? Same 
for other methods also



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchUtil.java
Lines 224 (patched)
<https://reviews.apache.org/r/71921/#comment308075>

Can we have curly braces here? It will avoid anyone make mistakes in the 
future wrt scope



security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchUtil.java
Lines 288 (patched)
<https://reviews.apache.org/r/71921/#comment308076>

Can we print other parameter from the method also? Same for other log 
messages in this method



security-admin/src/main/resources/conf.dist/ranger-admin-site.xml
Lines 49 (patched)
<https://reviews.apache.org/r/71921/#comment308077>

Could we have port also here?


- Don Bosco Durai


On Jan. 24, 2020, 7:56 p.m., And

Re: [REPORT] Apache Ranger - Jan 2020

[Don Bosco Durai]

+1 

Selva, thanks

Bosco


On 1/8/20, 12:47 PM, "Madhan Neethiraj"  wrote:

+1. Thanks Selva.

Madhan

On 1/8/20, 12:44 PM, "Velmurugan Periasamy"  wrote:

+1. Thanks Selva for putting together the report.



On Wed, Jan 8, 2020 at 2:54 PM Selvamohan Neethiraj 
 wrote:
All:

Here is the report for the Apache Board for the last quarter ending
Dec-2019.

Thanks,
Selva-

==>

## Description:
 - Apache Ranger is a framework to enable, monitor and manage 
comprehensive
   data security across the Hadoop platform

## Issues:
 - There are no issues requiring board attention at this time

## Activity:
 - Community has been working on 2.1.0 release

## Health report:
- Community is working on 2.1.0 release with
   - minor Fixes to 2.0.0 releases
   - performance improvements,
   - support for Hive 3.1.2
   - support for knox proxy for all plugins.

## PMC changes:
 - Currently 20 PMC members
 - No new PMC members. Last addition was Sailaja Polavarapu on 
2019-09-18.

## Committer base changes:
 - Currently 29 committers
 - No new committers. Last addition was Nikhil Purbhe on 2019-05-22.

## Releases:
 - Apache Ranger 2.0.0 was released on 2019-08-07.
 - Apache Ranger 1.2.0 was released on 2018-10-04.
 - Apache Ranger 1.1.0 was released on 2018-07-09.

## Mailing list activity:
 - Regular activity continues.
 - dev@ranger.apache.org :
- 879 emails sent to list (1106 in previous quarter)
 - u...@ranger.apache.org :
- 16 emails sent to list (15 in previous quarter)

## JIRA activity:
 - 65 JIRA tickets created in the last 3 months
 - 57 JIRA tickets closed/resolved in the last 3 months

Re: Review Request 71746: RANGER-2643: Use JPA/JDBC batching during policy creation

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71746/#review219034
---


Ship it!




Ship It!

- Don Bosco Durai


On Nov. 16, 2019, 10:46 p.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71746/
> ---
> 
> (Updated Nov. 16, 2019, 10:46 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-2643
> https://issues.apache.org/jira/browse/RANGER-2643
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Policy creation is slow if the policy contains a large number of users or 
> resources.  Part of the reason for this is that each row is a single 
> statement/transaction.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 7b2356bfd 
>   security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
> 3761ef2ce 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 85289dd71 
>   security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java 
> bdd8fbbb6 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> 54c9ee340 
>   security-admin/src/main/resources/META-INF/persistence.xml 211715978 
>   security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml 
> 99162971c 
> 
> 
> Diff: https://reviews.apache.org/r/71746/diff/2/
> 
> 
> Testing
> ---
> 
> Compilation/end-to-end test of policy creation.
> 
> 
> Thanks,
> 
> Andrew Luo
> 
>

Re: Review Request 71746: RANGER-2643: Use JPA/JDBC batching during policy creation

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71746/#review218592
---




security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java
Lines 46 (patched)
<https://reviews.apache.org/r/71746/#comment306354>

Should we take this value from propery? 

In RangerBizUtil, we already have a property for batchSize. But it might be 
better to create another one for this. E.g. bulkCreateBatchSize.

Otherwise, code looks good. Thanks for your contribution


- Don Bosco Durai


On Nov. 10, 2019, 8:09 p.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71746/
> ---
> 
> (Updated Nov. 10, 2019, 8:09 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-2643
> https://issues.apache.org/jira/browse/RANGER-2643
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Policy creation is slow if the policy contains a large number of users or 
> resources.  Part of the reason for this is that each row is a single 
> statement/transaction.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 7b2356bfd 
>   security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java 
> bdd8fbbb6 
>   security-admin/src/main/resources/META-INF/persistence.xml 211715978 
> 
> 
> Diff: https://reviews.apache.org/r/71746/diff/1/
> 
> 
> Testing
> ---
> 
> Compilation/end-to-end test of policy creation.
> 
> 
> Thanks,
> 
> Andrew Luo
> 
>

Re: [REPORT] Apache Ranger - Sep-2019

[Don Bosco Durai]

+1

Selva, thanks for putting this together.

Bosco


On 10/9/19, 5:50 AM, "Velmurugan Periasamy"  wrote:

+1. Thanks for putting it together. 



On Tue, Oct 8, 2019 at 4:48 PM Madhan Neethiraj  wrote:
+1 for the report.

Selva - thanks for compiling the report. Please update for a typo: siwtch 
=> switch.

Madhan



On 10/8/19, 9:38 AM, "Selvamohan Neethiraj" mailto:sneet...@apache.org>> wrote:

Apache Ranger PMC: 

Could you please review draft board report below and provide your 
feedback.


Thanks,
Selva-


## Description:
 - Apache Ranger is a framework to enable, monitor and manage 
comprehensive
   data security across the Hadoop platform

## Issues:
 - There are no issues requiring board attention at this time

## Activity:
 - Community has just released a major release - 2.0.0 and working on 
2.1.0 release.
 - Jira: has usual amount of activities in the community. +100(added)
   -72(resolved) over last 3 months

## Health report:
- Completed release of Apache Ranger 2.0.0 with major upgrade to 
support latest release of Hive, HBase and Hadoop.  
- Community is working on 2.1.0 release with
   - minor Fixes to 2.0.0 releases
   - Java 11 support
   - able to specify multiple ranger host (w/o load balancer)
   - siwtch logging framework to slf4j
 - Also, discussing new features for next major releases
   - authorization plugin for sparkSQL, ElasticSearch and Druid

## PMC changes:
 - Currently 20 PMC members
 - Sailaja Polavarapu was added to the PMC on 2019-09-18.

## Committer base changes:
 - Currently 29 committers
 - Nikhil Purbhe was added as a committer on Wed May 22 2019

## Releases:
 - Apache Ranger 2.0.0 was released on 2019-08-07.
 - Apache Ranger 1.2.0 was released on 2018-10-04.
 - Apache Ranger 1.1.0 was released on 2018-07-09.

## Mailing list activity:
 - Regular activity continues.
 - dev@ranger.apache.org :
- 1216 emails sent to list (905 in previous quarter)
 - u...@ranger.apache.org :
- 15 emails sent to list (43 in previous quarter)

## JIRA activity:
 - 100 JIRA tickets created in the last 3 months
 - 72 JIRA tickets closed/resolved in the last 3 months

Re: Review Request 71449: RANGER-2562 Support for configuration of multiple Solr URLs in the Solr service

[Don Bosco Durai]

> On Sept. 9, 2019, 3:16 p.m., Don Bosco Durai wrote:
> > plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrConnectionMgr.java
> > Line 33 (original), 35 (patched)
> > <https://reviews.apache.org/r/71449/diff/1/?file=2164383#file2164383line35>
> >
> > Instead of spliting the URLs and taking the last URL, can we support 
> > SolrCloud client using Zookeeper?
> 
> Kehua Wu wrote:
> Thank you for your reply!
> However, in some scenarios, SolrCloud does not require ZooKeeper to work 
> with. For example, my own test environment is just a simple SolrCloud mode 
> without Zookeeper, so I think the way Solr Url needs to be retained. But in 
> the future, I can open a new feature list to support the SolrCloud 
> configuration method of ZooKeeper.
> 
> Don Bosco Durai wrote:
> In your current implementation, you are connecting to the last Solr URL 
> in the list. So if that Solr instance is down, we won't be automatically 
> connecting to the other URLs. Correct me if I am wrong. I feel, the correct 
> implementation will be to use Zookeeper, else we have to modify our calling 
> code to use other Solr URLs when one of them fails with connection error.
> 
> Kehua Wu wrote:
> Thank you, but my implementation logic is to select the first connection 
> in the Solr URL to connect, if the connection fails, take the next one until 
> it succeeds.
> And the method "getSolrClient" will be called every time when testing 
> connection or get SolrCloud resources.
> 
> org.apache.ranger.services.solr.client.ServiceSolrConnectionMgr.getSolrClient(String,
>  Map)
> So every time Ranger connects to SolrCloud, it will try every Solr URL 
> connection until it succeeds.

I am not sure whether it works like the way you are envisioning. Can you test 
by bringing your Solr instance down in alternate order, but don't restart your 
Ranger.
Also, I was not aware you can have SolrCloud without Zookeeper. Can you point 
to the documentation which mentions how to create SolrCloud without Zookeeper. 
I can do a quick test. Thanks


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71449/#review217653
---


On Sept. 9, 2019, 8:23 a.m., Kehua Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71449/
> -------
> 
> (Updated Sept. 9, 2019, 8:23 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nitin Galave, pengjianhua, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, sam  rome, Venkat Ranganathan, Velmurugan 
> Periasamy, Qiang Zhang, and Barna Zsombor Klara.
> 
> 
> Bugs: RANGER-2562
> https://issues.apache.org/jira/browse/RANGER-2562
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> When create a new Solr service, we can set only one Solr URL, but as we know 
> Solr cloud model has more than one Solr service, so we should support for 
> configuration of multiple Solr URLs in the Solr service.
> 
> eg.
> 
> When I set "http://192.166.1.1:18983/solr; for "Solr URL" in the Solr 
> service, I click the "Test Connection" button, it will be ok, it shows 
> "Connected Successfully."
> 
> But when I set "http://192.166.1.1:18983/solr,http://192.166.1.2:28983/solr; 
> for "Solr URL" in the Solr service, I click the "Test Connection" button, it 
> will show "Connection Failed."
> 
> 
> Diffs
> -
> 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrConnectionMgr.java
>  f56373b 
> 
> 
> Diff: https://reviews.apache.org/r/71449/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kehua Wu
> 
>

Re: Review Request 71449: RANGER-2562 Support for configuration of multiple Solr URLs in the Solr service

[Don Bosco Durai]

> On Sept. 9, 2019, 3:16 p.m., Don Bosco Durai wrote:
> > plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrConnectionMgr.java
> > Line 33 (original), 35 (patched)
> > <https://reviews.apache.org/r/71449/diff/1/?file=2164383#file2164383line35>
> >
> > Instead of spliting the URLs and taking the last URL, can we support 
> > SolrCloud client using Zookeeper?
> 
> Kehua Wu wrote:
> Thank you for your reply!
> However, in some scenarios, SolrCloud does not require ZooKeeper to work 
> with. For example, my own test environment is just a simple SolrCloud mode 
> without Zookeeper, so I think the way Solr Url needs to be retained. But in 
> the future, I can open a new feature list to support the SolrCloud 
> configuration method of ZooKeeper.

In your current implementation, you are connecting to the last Solr URL in the 
list. So if that Solr instance is down, we won't be automatically connecting to 
the other URLs. Correct me if I am wrong. I feel, the correct implementation 
will be to use Zookeeper, else we have to modify our calling code to use other 
Solr URLs when one of them fails with connection error.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71449/#review217653
---


On Sept. 9, 2019, 8:23 a.m., Kehua Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71449/
> ---
> 
> (Updated Sept. 9, 2019, 8:23 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nitin Galave, pengjianhua, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, sam  rome, Venkat Ranganathan, Velmurugan 
> Periasamy, Qiang Zhang, and Barna Zsombor Klara.
> 
> 
> Bugs: RANGER-2562
> https://issues.apache.org/jira/browse/RANGER-2562
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> When create a new Solr service, we can set only one Solr URL, but as we know 
> Solr cloud model has more than one Solr service, so we should support for 
> configuration of multiple Solr URLs in the Solr service.
> 
> eg.
> 
> When I set "http://192.166.1.1:18983/solr; for "Solr URL" in the Solr 
> service, I click the "Test Connection" button, it will be ok, it shows 
> "Connected Successfully."
> 
> But when I set "http://192.166.1.1:18983/solr,http://192.166.1.2:28983/solr; 
> for "Solr URL" in the Solr service, I click the "Test Connection" button, it 
> will show "Connection Failed."
> 
> 
> Diffs
> -
> 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrConnectionMgr.java
>  f56373b 
> 
> 
> Diff: https://reviews.apache.org/r/71449/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kehua Wu
> 
>

Re: Review Request 71432: Configure Kerberos for Hive Ranger Client via HS2 configuration

[Don Bosco Durai]

> On Sept. 5, 2019, 3:42 p.m., Don Bosco Durai wrote:
> > hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
> > Lines 147 (patched)
> > <https://reviews.apache.org/r/71432/diff/1/?file=2163334#file2163334line147>
> >
> > What happens if the cluster is already Kerberos enabled?
> 
> Denys Kuzmenko wrote:
> Before the change, when the cluster was already Kerberos enabled, 
> MiscUtil.getUGILoginUser() delegated request to 
> UserGroupInformation.getLoginUser() as ugiLoginUser was never set.
> After the change it should start using ugiLoginUser.
> 
> public static UserGroupInformation getUGILoginUser()
> UserGroupInformation ret = ugiLoginUser;
> if (ret == null) {
> ret = UserGroupInformation.getLoginUser()
> }
> ...
> }
> 
> public ServicePolicies getServicePoliciesIfUpdated(...) {
> UserGroupInformation user = MiscUtil.getUGILoginUser();
> boolean isSecureMode = user != null && 
> UserGroupInformation.isSecurityEnabled();
> 
> if (isSecureMode) {
>   PrivilegedAction action = new 
> PrivilegedAction() {
> public ClientResponse run() {
>   WebResource secureWebResource = 
> RangerAdminRESTClient.this.createWebResource("/service/plugins/secure/ ...);
>   return (ClientResponse)secureWebResource.accept(new 
> String[]{"application/json"}).get(ClientResponse.class);
> }
>   };
>   ...
> }

Since it was working before this change, do you think calling this method will 
have side affect? In an existing Kerberos Hive, we rely on Hive Server2 to 
manage the UGI, right? Ideally, we shouldn't change static variables managed by 
the component. If we do, let's make sure there are no side affects.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71432/#review217591
---


On Sept. 5, 2019, 12:13 p.m., Denys Kuzmenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71432/
> ---
> 
> (Updated Sept. 5, 2019, 12:13 p.m.)
> 
> 
> Review request for ranger and Ramesh Mani.
> 
> 
> Bugs: RANGER-2557
> https://issues.apache.org/jira/browse/RANGER-2557
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> In Hive we would like to have possibility to enable Kerberos partially (i.e 
> only Ranger, Atlas and HMS).
> However, since hadoop security is a global flag there are many places that 
> need to be commented out to avoid the UGI cluster wide configuration.
> 
> 
> Diffs
> -
> 
>   agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
> b7315a922 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  bb015c595 
> 
> 
> Diff: https://reviews.apache.org/r/71432/diff/1/
> 
> 
> Testing
> ---
> 
> On local cluster.
> 
> 
> Thanks,
> 
> Denys Kuzmenko
> 
>

Re: Review Request 71449: RANGER-2562 Support for configuration of multiple Solr URLs in the Solr service

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71449/#review217653
---




plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrConnectionMgr.java
Line 33 (original), 35 (patched)
<https://reviews.apache.org/r/71449/#comment304940>

Instead of spliting the URLs and taking the last URL, can we support 
SolrCloud client using Zookeeper?


- Don Bosco Durai


On Sept. 9, 2019, 8:23 a.m., Kehua Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71449/
> ---
> 
> (Updated Sept. 9, 2019, 8:23 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nitin Galave, pengjianhua, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, sam  rome, Venkat Ranganathan, Velmurugan 
> Periasamy, Qiang Zhang, and Barna Zsombor Klara.
> 
> 
> Bugs: RANGER-2562
> https://issues.apache.org/jira/browse/RANGER-2562
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> When create a new Solr service, we can set only one Solr URL, but as we know 
> Solr cloud model has more than one Solr service, so we should support for 
> configuration of multiple Solr URLs in the Solr service.
> 
> eg.
> 
> When I set "http://192.166.1.1:18983/solr; for "Solr URL" in the Solr 
> service, I click the "Test Connection" button, it will be ok, it shows 
> "Connected Successfully."
> 
> But when I set "http://192.166.1.1:18983/solr,http://192.166.1.2:28983/solr; 
> for "Solr URL" in the Solr service, I click the "Test Connection" button, it 
> will show "Connection Failed."
> 
> 
> Diffs
> -
> 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrConnectionMgr.java
>  f56373b 
> 
> 
> Diff: https://reviews.apache.org/r/71449/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kehua Wu
> 
>

Re: Review Request 71438: RANGER-2560 Solve the problem of the order of the configuration items of the Solr plugin

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71438/#review217590
---


Ship it!




Ship It!

- Don Bosco Durai


On Sept. 5, 2019, 12:12 p.m., Kehua Wu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71438/
> ---
> 
> (Updated Sept. 5, 2019, 12:12 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nitin Galave, pengjianhua, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, sam  rome, Venkat Ranganathan, Velmurugan 
> Periasamy, Qiang Zhang, and Barna Zsombor Klara.
> 
> 
> Bugs: RANGER-2560
> https://issues.apache.org/jira/browse/RANGER-2560
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> There is one problem in the method "init()" of class "RangerSolrAuthorizer".
> 
> Firstly, the code logic is to get the parameters "useProxyIP", 
> "useProxyIP","solrAppName" and "solrAppName" from RangerConfiguration, as 
> follows:
> 
> useProxyIP = RangerConfiguration.getInstance().getBoolean(useProxyIP = 
> RangerConfiguration.getInstance().getBoolean( PROP_USE_PROXY_IP, useProxyIP);
> proxyIPHeader = RangerConfiguration.getInstance().get( PROP_PROXY_IP_HEADER, 
> proxyIPHeader);
> // First get from the -D property
> solrAppName = System.getProperty("solr.kerberos.jaas.appname", solrAppName);
> // Override if required from Ranger properties
> solrAppName = RangerConfiguration.getInstance().get( PROP_SOLR_APP_NAME, 
> solrAppName);
> But after that, the code logic is to call "solrPlugin.init()" to parse the 
> configuration file, as follows:
> 
> configuration.addResourcesForServiceType(serviceType); 
> Due to the opposite logic, the values of the parameters "useProxyIP", 
> "useProxyIP","solrAppName" and "solrAppName" are not available.
> 
> 
> Diffs
> -
> 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>  f87e531 
> 
> 
> Diff: https://reviews.apache.org/r/71438/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kehua Wu
> 
>

[jira] [Commented] (RANGER-2551) Optimize obtaining the agentHostname of audit log

[Don Bosco Durai (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918189#comment-16918189
 ] 

Don Bosco Durai commented on RANGER-2551:
-

[~Seymour Xu] which Ranger version are you using? The stack trace in the JIRA 
is not matching the code. In the last MiscUtil.getHostname() we are already 
taking care of caching the hostname.
{code:java}
public static String getHostname() {
String ret = local_hostname;
if  (ret == null) {
initLocalHost();
ret = local_hostname;
if (ret == null) {
ret = "unknown";
}
}
return ret;
}
{code}


> Optimize obtaining the agentHostname of audit log 
> --
>
> Key: RANGER-2551
> URL: https://issues.apache.org/jira/browse/RANGER-2551
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins, Ranger
>Affects Versions: 1.1.0, 2.0.0, 1.2.0
>Reporter: Haihui Xu
>Assignee: Haihui Xu
>Priority: Major
>  Labels: patch
> Fix For: 2.1.0
>
> Attachments: RANGER-2551.patch
>
>
> Firstly, kafka enable ranger-kafka-plugin,. The ranger audit log event 
> happens very frequently when the kafka client access kafka broker, thus 
> affectting kafka  performance.  The jstack of kakfa broker pid logs is:
> "kafka-request-handler-23" #101 daemon prio=5 os_prio=0 
> tid=0x7fbcce7d5000 nid=0x3331a runnable [0x7fb356e3000]
>  java.lang.Thread.State: RUNNABLE
> at java.net.Inet4AddressImpl.getLocalHostName(Native Method)
> at java.net.InetAddress.getLocalHost(InetAddress.java:1474)
> at 
> org.apache.ranger.audit.provider.MiscUtil.getHostname(MiscUtil.java 166)
> at 
> org.apache.ranger.plugin.audit.RangerDefaultAuditHandler.populateDefaults(RangerDefaultAuditHandler.java:198)
> at 
> org.apache.ranger.plugin.audit.RangerDefaultAuditHandler.getAuthezEvents(RangerDefaultAuditHandler.java:132)



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (RANGER-2551) Optimize obtaining the agentHostname of audit log

[Don Bosco Durai (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915597#comment-16915597
 ] 

Don Bosco Durai commented on RANGER-2551:
-

[~Seymour Xu], the patch looks good. Can you create a Review Board review?  
Thanks

> Optimize obtaining the agentHostname of audit log 
> --
>
> Key: RANGER-2551
> URL: https://issues.apache.org/jira/browse/RANGER-2551
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins, Ranger
>Affects Versions: 1.1.0, 2.0.0, 1.2.0
>Reporter: Haihui Xu
>Assignee: Haihui Xu
>Priority: Major
>  Labels: patch
> Fix For: 2.1.0
>
> Attachments: RANGER-2551.patch
>
>
> Firstly, kafka enable ranger-kafka-plugin,. The ranger audit log event 
> happens very frequently when the kafka client access kafka broker, thus 
> affectting kafka  performance.  The jstack of kakfa broker pid logs is:
> "kafka-request-handler-23" #101 daemon prio=5 os_prio=0 
> tid=0x7fbcce7d5000 nid=0x3331a runnable [0x7fb356e3000]
>  java.lang.Thread.State: RUNNABLE
> at java.net.Inet4AddressImpl.getLocalHostName(Native Method)
> at java.net.InetAddress.getLocalHost(InetAddress.java:1474)
> at 
> org.apache.ranger.audit.provider.MiscUtil.getHostname(MiscUtil.java 166)
> at 
> org.apache.ranger.plugin.audit.RangerDefaultAuditHandler.populateDefaults(RangerDefaultAuditHandler.java:198)
> at 
> org.apache.ranger.plugin.audit.RangerDefaultAuditHandler.getAuthezEvents(RangerDefaultAuditHandler.java:132)



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Re: (无主题)

[Don Bosco Durai]

Thanks for your interest in Apache Ranger. I have added you as contributor.

Regards

Bosco


On 8/22/19, 7:07 PM, "wkh8011"  wrote:

Hi Guys,


I want to contribute to Apache Ranger.
Would you please give me the permission as a contributor?
My JIRA ID is wkh8011.
Thanks.


| |
wkh8011
|
|
邮箱：wkh8...@163.com
|

Signature is customized by Netease Mail Master

[jira] [Commented] (RANGER-2541) Upgrade Ranger to support Elasticsearch 7.x

[Don Bosco Durai (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912026#comment-16912026
 ] 

Don Bosco Durai commented on RANGER-2541:
-

Should we consider supporting Elastic Search from AWS also?

> Upgrade Ranger to support Elasticsearch 7.x
> ---
>
> Key: RANGER-2541
> URL: https://issues.apache.org/jira/browse/RANGER-2541
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Major
>
> We should upgrade Elasticsearch to a more recent release. 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Kafka Authorizer API changes - KIP-504

[Don Bosco Durai]

The Apache Kafka community is proposing a new interface for Kafka Topic 
authorization. They are going to continue supporting the older interface, but 
it will be deprecated soon. The new interface doesn’t affect much to us. I have 
reviewed the KIP and have also given my feedback. Feel free to review and 
provide your feedback either to the Kafka mailing list or here (and I can take 
it there).

 

Few key points:
It is going to be Java interface, which makes it is easy for us
Additional context in the request
Support multiple listeners and pass on the listener context as part of request. 
E.g. inter broker communication, etc…
Hints for audit logging  (e.g. where the request is for check listing, etc.). 
We can decide what to log to audit destination. This will reduce the number of 
logs
Count for similar requests. Similar to what we are already doing in 
batching/aggregating audit count. This will make it easier and reduce CPU 
cycles on our side.
Authorize method could have multiple requests. We need to treat them 
accordingly. 
There are few additional classes for abstraction.
 

KIP-504 detail

https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface

 

Discussion thread 

https://lists.apache.org/thread.html/f9830e4cb4bd7e9cc031c51395dfd670ec6839fef432d86d5074334b@%3Cdev.kafka.apache.org%3E

 

Thanks

 

Bosco

 

Re: Review Request 70981: RANGER-2494:Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70981/#review216265
---




agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsNotPresentConditionEvaluator.java
Lines 67 (patched)
<https://reviews.apache.org/r/70981/#comment303398>

The braces are not matching


- Don Bosco Durai


On July 1, 2019, 6:37 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70981/
> ---
> 
> (Updated July 1, 2019, 6:37 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2494
> https://issues.apache.org/jira/browse/RANGER-2494
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2494:Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsAnyPresentConditionEvaluator.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsNotPresentConditionEvaluator.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70981/diff/2/
> 
> 
> Testing
> ---
> 
> Testing done in LOCAL VM
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 70981: RANGER-2494:Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent

[Don Bosco Durai]

> On July 1, 2019, 2:19 a.m., Ramesh Mani wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsNotPresentConditionEvaluator.java
> > Lines 72 (patched)
> > <https://reviews.apache.org/r/70981/diff/1/?file=2152911#file2152911line72>
> >
> > when resourceTags will be null here.
> > 
> > Set resourceTags= context.getAllTagTypes()  going to give 
> > either null or set of resource tags.
> > So in that case when null we don't allow.
> > I shall check for empty also here so it wont be confusing.

Ideally, it should be consistent. I not sure when we would get null. If it is 
error condition, then not allowing is fine. But if it could be null because 
there are no tags, then we should allow, because the condition says "Tags not 
present", so empty tag list means the tags are not present.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70981/#review216256
---


On July 1, 2019, 1:03 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70981/
> -------
> 
> (Updated July 1, 2019, 1:03 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2494
> https://issues.apache.org/jira/browse/RANGER-2494
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2494:Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsAnyPresentConditionEvaluator.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsNotPresentConditionEvaluator.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70981/diff/1/
> 
> 
> Testing
> ---
> 
> Testing done in LOCAL VM
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

[jira] [Commented] (RANGER-2494) Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875901#comment-16875901
 ] 

Don Bosco Durai commented on RANGER-2494:
-

[~rmani] thanks for the clarification. These are good enhancements. Thanks

> Ranger Custom PolicyCondition for TagsNotPresent and  AnyTagPresent
> ---
>
> Key: RANGER-2494
> URL: https://issues.apache.org/jira/browse/RANGER-2494
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master, 2.0.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-2494-Ranger-Custom-PolicyCondition-for-TagNot.patch
>
>
> Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent.
> - Two new Custom Policy Conditions are to be created.
> 1) RangerTagsAnyPresentConditionEvaluator - This condition evaluates to 
> "true" when  "Any of the Policy condition Tag" defined is present  in the 
> tags associated to the Resource.
> 2) RangerTagsNotPresentConditionEvaluator - This condition evaluates to 
> "true" when "None of the Policy condition Tag" defined is present in the tags 
> associated to the resource.
> These new custom policy conditions are in addition to the one created in 
> https://issues.apache.org/jira/browse/RANGER-2465



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70981: RANGER-2494:Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70981/#review216255
---




agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsNotPresentConditionEvaluator.java
Lines 72 (patched)
<https://reviews.apache.org/r/70981/#comment303384>

Seems confusing. What if resourceTags is null? Do we allow or not? 
According to this, if it is null we don't allow, but if empty we will allow. I 
might be wrong.


- Don Bosco Durai


On July 1, 2019, 1:03 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70981/
> ---
> 
> (Updated July 1, 2019, 1:03 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2494
> https://issues.apache.org/jira/browse/RANGER-2494
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2494:Ranger Custom PolicyCondition for TagsNotPresent and AnyTagPresent
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsAnyPresentConditionEvaluator.java
>  PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerTagsNotPresentConditionEvaluator.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70981/diff/1/
> 
> 
> Testing
> ---
> 
> Testing done in LOCAL VM
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

[jira] [Commented] (RANGER-2494) Ranger Custom PolicyCondition for TagNotPresent and AnyTagPresent

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875897#comment-16875897
 ] 

Don Bosco Durai commented on RANGER-2494:
-

[~rmani] can you give additional information on this JIRA?

Thanks

> Ranger Custom PolicyCondition for TagNotPresent and  AnyTagPresent
> --
>
> Key: RANGER-2494
> URL: https://issues.apache.org/jira/browse/RANGER-2494
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master, 2.0.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-2494-Ranger-Custom-PolicyCondition-for-TagNot.patch
>
>
> Ranger Custom PolicyCondition for TagNotPresent and  AnyTagPresent



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70893/#review216247
---


Ship it!




Ship It!

- Don Bosco Durai


On June 29, 2019, 5:50 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70893/
> ---
> 
> (Updated June 29, 2019, 5:50 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh 
> Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2477
> https://issues.apache.org/jira/browse/RANGER-2477
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is 
> enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger 
> expects that x-forwarded-host shall be provided by LB so current 
> implementation forward the request to the same host but does not change the 
> protocol to https if LB is also SSL(x-forwarded-proto)
> 
> Proposed solution: proposed patch contains changes which shall replace the 
> x-forwarded-proto value in the request URL if request URL  contains protocol 
> http while x-forwarded-proto value is https.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
>  c3fbe9c23 
> 
> 
> Diff: https://reviews.apache.org/r/70893/diff/3/
> 
> 
> Testing
> ---
> 
> Tested knoxsso, knox proxy and ranger HA based authentications.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70893/#review216245
---




security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
Lines 281 (patched)
<https://reviews.apache.org/r/70893/#comment303375>

To be on the safe side, you might want to consider using relaceFirst().


- Don Bosco Durai


On June 29, 2019, 5:04 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70893/
> ---
> 
> (Updated June 29, 2019, 5:04 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh 
> Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2477
> https://issues.apache.org/jira/browse/RANGER-2477
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is 
> enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger 
> expects that x-forwarded-host shall be provided by LB so current 
> implementation forward the request to the same host but does not change the 
> protocol to https if LB is also SSL(x-forwarded-proto)
> 
> Proposed solution: proposed patch contains changes which shall replace the 
> x-forwarded-proto value in the request URL if request URL  contains protocol 
> http while x-forwarded-proto value is https.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
>  c3fbe9c23 
> 
> 
> Diff: https://reviews.apache.org/r/70893/diff/2/
> 
> 
> Testing
> ---
> 
> Tested knoxsso, knox proxy and ranger HA based authentications.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70893/#review216244
---


Ship it!




Ship It!

- Don Bosco Durai


On June 19, 2019, 4:22 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70893/
> ---
> 
> (Updated June 19, 2019, 4:22 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh 
> Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2477
> https://issues.apache.org/jira/browse/RANGER-2477
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is 
> enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger 
> expects that x-forwarded-host shall be provided by LB so current 
> implementation forward the request to the same host but does not change the 
> protocol to https if LB is also SSL(x-forwarded-proto)
> 
> Proposed solution: proposed patch contains changes which shall replace the 
> x-forwarded-proto value in the request URL if request URL  contains protocol 
> http while x-forwarded-proto value is https.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
>  c3fbe9c23 
> 
> 
> Diff: https://reviews.apache.org/r/70893/diff/1/
> 
> 
> Testing
> ---
> 
> Tested knoxsso, knox proxy and ranger HA based authentications.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded

[Don Bosco Durai]

> On June 29, 2019, 2:58 a.m., Don Bosco Durai wrote:
> > security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
> > Lines 280 (patched)
> > <https://reviews.apache.org/r/70893/diff/1/?file=2151380#file2151380line280>
> >
> > Just curios, what happens if the request URL is https and 
> > xForwardedProt is http? Is it a valid combination?
> > 
> > Also, any reason, we are not checking just for "http:"? Instead, 2 
> > conditions?
> 
> Pradeep Agrawal wrote:
> =>I don't think its a valid combination but it may happen.
> Usually, xForwardedProto is either provided as header by user or it can 
> be overridden by proxy/load-balancer. if its decided by proxy/load-balancer 
> then it will be according to the request but user may make a mistake by 
> putting http rather actually https is needed. 
> line 283 will make the forwardURL similar to the requestURL and will 
> ignore the xForwardedProto value.
> 
> 
> => When load balancer is in https and ranger is in http and knoxSSO is 
> enabled and if x-forwarded-host is not provided then 
> we can assume that the request can be forwarded to the same host from 
> where the request is coming 
> here though LB is in ssl, received requestURL was in http(bit strange may 
> be LB issue probably similar to 
> https://stackoverflow.com/questions/29469929/why-does-request-getrequesturl-return-non-https-url)
> so to handle this situation I am considering xForwardedProto value which 
> was https so replacing http with https.
> since i am using startsWith() method and https starts with http so just 
> for http case i need to add extra condition here as i want to replace http 
> only and avoid wrong replaces like https -> httpss
> if requestURL contains https then line 283 shall make the requestURL to 
> be a forwardURL.
> 
> 
> If there is a better way to handle this please advice.

Pradeep, thanks for your explanation. Regarding the http check, I was seeing 
whether we can check only for starts with "http:", rather than starts with 
"http" and not "https". The net effect would be the same.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70893/#review216240
---


On June 19, 2019, 4:22 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70893/
> ---
> 
> (Updated June 19, 2019, 4:22 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh 
> Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2477
> https://issues.apache.org/jira/browse/RANGER-2477
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is 
> enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger 
> expects that x-forwarded-host shall be provided by LB so current 
> implementation forward the request to the same host but does not change the 
> protocol to https if LB is also SSL(x-forwarded-proto)
> 
> Proposed solution: proposed patch contains changes which shall replace the 
> x-forwarded-proto value in the request URL if request URL  contains protocol 
> http while x-forwarded-proto value is https.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
>  c3fbe9c23 
> 
> 
> Diff: https://reviews.apache.org/r/70893/diff/1/
> 
> 
> Testing
> ---
> 
> Tested knoxsso, knox proxy and ranger HA based authentications.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70893/#review216240
---




security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
Lines 280 (patched)
<https://reviews.apache.org/r/70893/#comment303371>

Just curios, what happens if the request URL is https and xForwardedProt is 
http? Is it a valid combination?

Also, any reason, we are not checking just for "http:"? Instead, 2 
conditions?


- Don Bosco Durai


On June 19, 2019, 4:22 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70893/
> ---
> 
> (Updated June 19, 2019, 4:22 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh 
> Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2477
> https://issues.apache.org/jira/browse/RANGER-2477
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is 
> enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger 
> expects that x-forwarded-host shall be provided by LB so current 
> implementation forward the request to the same host but does not change the 
> protocol to https if LB is also SSL(x-forwarded-proto)
> 
> Proposed solution: proposed patch contains changes which shall replace the 
> x-forwarded-proto value in the request URL if request URL  contains protocol 
> http while x-forwarded-proto value is https.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
>  c3fbe9c23 
> 
> 
> Diff: https://reviews.apache.org/r/70893/diff/1/
> 
> 
> Testing
> ---
> 
> Tested knoxsso, knox proxy and ranger HA based authentications.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Created] (RANGER-2471) Service Def only support 127 permissions

[Don Bosco Durai (JIRA)]

Don Bosco Durai created RANGER-2471:
---

 Summary: Service Def only support 127 permissions 
 Key: RANGER-2471
 URL: https://issues.apache.org/jira/browse/RANGER-2471
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Don Bosco Durai


Service Def contains the list of permissions it supports, e.g. Select, Insert 
in Hive. The DB field we are using storing the sort order 
(x_access_type_def::sort_order) is currently tinyint(3), which limits the 
number of permissions supported to around 128. 

While this works for most of the service definitions, but it puts a limitation 
for the common service def used by "Tags", which is generally a cumulative of 
all the permissions.

We need to change the column to a DB type which can support more values. e.g. 
smallint




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (RANGER-2470) Support for Okta Authentication for Ranger Admin Consol

[Don Bosco Durai (JIRA)]

Don Bosco Durai created RANGER-2470:
---

 Summary: Support for Okta Authentication for Ranger Admin Consol
 Key: RANGER-2470
 URL: https://issues.apache.org/jira/browse/RANGER-2470
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Don Bosco Durai


Currently we support SSO via Knox. It might be good to also support SSO without 
Knox dependency.

I have created this JIRA with Okta in mind, but from the design point of view, 
we should plan to support SAML and possibly OAuth/OpenId



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2461) ranger-kafka-plugin find resource bug

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16864379#comment-16864379
 ] 

Don Bosco Durai commented on RANGER-2461:
-

Glad your classpath issue got resolved.

1. Regarding Kerberos, just want to make sure that both Ranger and Kafka is 
Kerberos enabled
2. Can I assume, without Ranger you werre able to connect to Kafka Broker 
**only** after doing kinit and were able to publish/consume



> ranger-kafka-plugin find resource bug
> -
>
> Key: RANGER-2461
> URL: https://issues.apache.org/jira/browse/RANGER-2461
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 1.2.0
>Reporter: Konstantin Tsypin
>Priority: Major
>
> Hi there
> Past several days i tried to implement integration apache ranger and 
> ranger-kafka plugin.
> *My versions:*
> kafka: 2.12_2.2.0
> ranger: 1.2.0 , ranger-plugin 1.2.0
> *My current problem:*
> as far as i understand, kafka cant find the ranger-created audit files that i 
> made with enable-kafka-plugin.sh script (ranger-kafka-audit.xml and 
> ranger-kafka-security.xml). I sold this files everywhere in file system but 
> this class still cant find it and cause fatal error:
>  
> [2019-06-06 13:22:44,584] INFO 
> getFilesInDirectory('/usr/lib/kafka/libs/ranger-kafka-plugin-impl'): adding 
> /usr/lib/kafka/libs/ranger-kafka-plugin-impl/guava-17.0.jar 
> (org.apache.ranger.plugin.classloader.RangerPluginClassLoaderUtil)
>  [2019-06-06 13:22:44,584] INFO 
> getFilesInDirectory('/usr/lib/kafka/libs/ranger-kafka-plugin-impl'): adding 
> /usr/lib/kafka/libs/ranger-kafka-plugin-impl/ranger-plugins-common-1.2.0.jar 
> (org.apache.ranger.plugin.classloader.RangerPluginClassLoaderUtil)
>  [2019-06-06 13:22:44,615] INFO [Transaction Marker Channel Manager 0]: 
> Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
>  [2019-06-06 13:22:44,648] ERROR Error getting principal. 
> (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>  java.lang.NoSuchMethodError: 
> org.apache.kafka.common.security.JaasContext.load(Lorg/apache/kafka/common/security/JaasContext$Type;Lorg/apache/kafka/common/network/ListenerName;Ljava/util/Map;)Lorg/apache/kafka/common/security/JaasContext;
>  at 
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:98)
>  at 
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:94)
>  at kafka.server.KafkaServer.$anonfun$startup$5(KafkaServer.scala:288)
>  at scala.Option.map(Option.scala:163)
>  at kafka.server.KafkaServer.startup(KafkaServer.scala:286)
>  at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
>  at kafka.Kafka$.main(Kafka.scala:75)
>  at kafka.Kafka.main(Kafka.scala)
>  [2019-06-06 13:22:44,689] INFO Calling plugin.init() 
> (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>  [2019-06-06 13:22:44,733] ERROR 
> addResourceIfReadable(ranger-kafka-audit.xml): couldn't find resource file 
> location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
>  [2019-06-06 13:22:44,734] ERROR 
> addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file 
> location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
>  [2019-06-06 13:22:44,758] INFO AuditProviderFactory: creating.. 
> (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,758] INFO AuditProviderFactory: initializing.. 
> (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,900] INFO No v3 audit configuration found. Trying v2 
> audit configurations (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,900] INFO AuditProviderFactory: Audit not enabled.. 
> (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,900] INFO PolicyEngineOptions: \{ evaluatorType: auto, 
> cacheAuditResult: false, disableContextEnrichers: false, 
> disableCustomConditions: false, disableTrieLookupPrefilter: false, 
> optimizeTrieForRetrieval: false } 
> (org.apache.ranger.plugin.service.RangerBasePlugin)
>  [2019-06-06 13:22:44,917] ERROR [KafkaServer id=0] Fatal error during 
> KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
>  java.lang.NullPointerException
>  
> [root@myhost kafka]# find / -name ranger-kafka-security.xml | xargs ls -lah {}
>  ls: cannot access {}: No such file or directory
>  -rwxr--r-- 1 root root 2.9K Jun 5 20:41 
> /opt/kafka/conf/ranger-kafka

[jira] [Commented] (RANGER-2461) ranger-kafka-plugin find resource bug

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861915#comment-16861915
 ] 

Don Bosco Durai commented on RANGER-2461:
-

I have to look into this more carefully. I am not sure whether you updated the 
classpath
For now try this:

#bin/kafka-server-start.sh
#Add the below lines before exec $base_dir/kafka-run-class.sh $EXTRA_ARGS 
kafka.Kafka "$@"

classpathmunge /home/ec2-user/kafka_2.12-2.2.0/config
#classpathmunge '/usr/hdp/current/hadoop-hdfs-client/*' 

#classpathmunge '/usr/hdp/current/hadoop-hdfs-client/lib/*' 

#classpathmunge '/etc/hadoop/conf'  

export CLASSPATH
unset classpathmunge

#Update hdfs jar path and conf as needed.



> ranger-kafka-plugin find resource bug
> -
>
> Key: RANGER-2461
> URL: https://issues.apache.org/jira/browse/RANGER-2461
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 1.2.0
>Reporter: Konstantin Tsypin
>Priority: Major
>
> Hi there
> Past several days i tried to implement integration apache ranger and 
> ranger-kafka plugin.
> *My versions:*
> kafka: 2.12_2.2.0
> ranger: 1.2.0 , ranger-plugin 1.2.0
> *My current problem:*
> as far as i understand, kafka cant find the ranger-created audit files that i 
> made with enable-kafka-plugin.sh script (ranger-kafka-audit.xml and 
> ranger-kafka-security.xml). I sold this files everywhere in file system but 
> this class still cant find it and cause fatal error:
>  
> [2019-06-06 13:22:44,584] INFO 
> getFilesInDirectory('/usr/lib/kafka/libs/ranger-kafka-plugin-impl'): adding 
> /usr/lib/kafka/libs/ranger-kafka-plugin-impl/guava-17.0.jar 
> (org.apache.ranger.plugin.classloader.RangerPluginClassLoaderUtil)
>  [2019-06-06 13:22:44,584] INFO 
> getFilesInDirectory('/usr/lib/kafka/libs/ranger-kafka-plugin-impl'): adding 
> /usr/lib/kafka/libs/ranger-kafka-plugin-impl/ranger-plugins-common-1.2.0.jar 
> (org.apache.ranger.plugin.classloader.RangerPluginClassLoaderUtil)
>  [2019-06-06 13:22:44,615] INFO [Transaction Marker Channel Manager 0]: 
> Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
>  [2019-06-06 13:22:44,648] ERROR Error getting principal. 
> (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>  java.lang.NoSuchMethodError: 
> org.apache.kafka.common.security.JaasContext.load(Lorg/apache/kafka/common/security/JaasContext$Type;Lorg/apache/kafka/common/network/ListenerName;Ljava/util/Map;)Lorg/apache/kafka/common/security/JaasContext;
>  at 
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:98)
>  at 
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:94)
>  at kafka.server.KafkaServer.$anonfun$startup$5(KafkaServer.scala:288)
>  at scala.Option.map(Option.scala:163)
>  at kafka.server.KafkaServer.startup(KafkaServer.scala:286)
>  at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
>  at kafka.Kafka$.main(Kafka.scala:75)
>  at kafka.Kafka.main(Kafka.scala)
>  [2019-06-06 13:22:44,689] INFO Calling plugin.init() 
> (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
>  [2019-06-06 13:22:44,733] ERROR 
> addResourceIfReadable(ranger-kafka-audit.xml): couldn't find resource file 
> location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
>  [2019-06-06 13:22:44,734] ERROR 
> addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file 
> location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
>  [2019-06-06 13:22:44,758] INFO AuditProviderFactory: creating.. 
> (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,758] INFO AuditProviderFactory: initializing.. 
> (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,900] INFO No v3 audit configuration found. Trying v2 
> audit configurations (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,900] INFO AuditProviderFactory: Audit not enabled.. 
> (org.apache.ranger.audit.provider.AuditProviderFactory)
>  [2019-06-06 13:22:44,900] INFO PolicyEngineOptions: \{ evaluatorType: auto, 
> cacheAuditResult: false, disableContextEnrichers: false, 
> disableCustomConditions: false, disableTrieLookupPrefilter: false, 
> optimizeTrieForRetrieval: false } 
> (org.apache.ranger.plugin.servi

Re: Review Request 70817: RANGER-2465:Create a PolicyCondition to apply if all given tags are present for the accessed resource

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70817/#review215803
---


Ship it!




Ship It!

- Don Bosco Durai


On June 11, 2019, 4:28 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70817/
> ---
> 
> (Updated June 11, 2019, 4:28 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2465:Create a PolicyCondition to apply if all given tags are present 
> for the accessed resource
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerMultipleTagsConditionEvaluator.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70817/diff/2/
> 
> 
> Testing
> ---
> 
> Tested in Local VM.
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Recent Installation Guide / Version Matrix

[Don Bosco Durai]

John, there was already issue created for the Docker script. Seems you have 
already worked around.

Anyway, I have uploaded the patch with the fix for gosu related issues.

https://reviews.apache.org/r/70825/

https://issues.apache.org/jira/browse/RANGER-2410

Bosco


On 6/9/19, 4:28 PM, "John Humphreys"  wrote:

Thanks for the reply Don; that all makes sense :).

I did manage to build the 1.2 code using the docker build script (with some
minor tweaks for a missing maven download and some gosu issues).

I've put my code on hold for the moment as I found out that Presto won't
get column-level permissioning through hive.  I see some new commits with a
Presto plugin made about 20 days ago, so I guess I'll be waiting for that.
I'll check out the related JIRAs/etc.

Thanks again,

-John

On Sun, Jun 9, 2019 at 3:51 PM Don Bosco Durai  wrote:

> Hi John
>
> We need to improve on the documentation. Currently, most of these
> information are in pom files in each release. With so many asking the same
> questions, we should plan to have this as part of the document from our
> next release onwards. Also the installation documentation needs an 
overhaul.
>
> For the time being, Ranger 1.2 release does support Hive 2.x. Let us know
> if you face any issue. We can create a JIRA to start tracking the update 
to
> the documentation.
>
> Thanks
>
> Bosco
>
>
> On 6/8/19, 2:21 PM, "John Humphreys"  wrote:
>
> Hi everyone,
>
> I'm working to install Ranger (just in our own hive for use with
> Presto/etc).  So, I'm not on HDP or anything.
>
> The most recent install guide I can find is for version 0.5.0
>
> 
https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation
> .
>
> Also, this guide states that only hive 1.2.0 is supported.  But 
clearly
> from JIRAs like this:
> https://issues.apache.org/jira/browse/RANGER-1927,
> Hive 2.x is supported.
>
> Is there a better place to find up to date installation information 
and
> compatibility information?  I'm really excited to use Ranger but it
> seems
> to be hard outside of HDP unless I'm missing something.
>
> Thanks,
>
> -John Humphreys
>
>
>
>

Review Request 70825: Removed dependencies with gosu in Docker build script

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70825/
---

Review request for ranger.


Bugs: RANGER-2410
https://issues.apache.org/jira/browse/RANGER-2410


Repository: ranger


Description
---

The previous build script using docker had dependencies with gosu. Updated to 
remove dependencies. Also updated maven version


Diffs
-

  build_ranger_using_docker.sh 17c8ed0ca 


Diff: https://reviews.apache.org/r/70825/diff/1/


Testing
---

Tested in local env and also testing Helene Treadwell.


Thanks,

Don Bosco Durai

Re: Recent Installation Guide / Version Matrix

[Don Bosco Durai]

Hi John

We need to improve on the documentation. Currently, most of these information 
are in pom files in each release. With so many asking the same questions, we 
should plan to have this as part of the document from our next release onwards. 
Also the installation documentation needs an overhaul.

For the time being, Ranger 1.2 release does support Hive 2.x. Let us know if 
you face any issue. We can create a JIRA to start tracking the update to the 
documentation.

Thanks

Bosco


On 6/8/19, 2:21 PM, "John Humphreys"  wrote:

Hi everyone,

I'm working to install Ranger (just in our own hive for use with
Presto/etc).  So, I'm not on HDP or anything.

The most recent install guide I can find is for version 0.5.0

https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5.0+Installation
.

Also, this guide states that only hive 1.2.0 is supported.  But clearly
from JIRAs like this: https://issues.apache.org/jira/browse/RANGER-1927,
Hive 2.x is supported.

Is there a better place to find up to date installation information and
compatibility information?  I'm really excited to use Ranger but it seems
to be hard outside of HDP unless I'm missing something.

Thanks,

-John Humphreys

[jira] [Commented] (RANGER-2395) Add presto plugin

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16843299#comment-16843299
 ] 

Don Bosco Durai commented on RANGER-2395:
-

[~bolke] I have given you the permission to the Ranger Wiki

> Add presto plugin
> -
>
> Key: RANGER-2395
> URL: https://issues.apache.org/jira/browse/RANGER-2395
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Bolke de Bruin
>Assignee: Bolke de Bruin
>Priority: Major
> Fix For: master
>
> Attachments: 0001-Add-Presto-plugin.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Presto (or PrestoDB) is an open source, distributed SQL query engine, 
> designed from the ground up for fast analytic queries against data of any 
> size. It supports both non-relational sources, such as the Hadoop Distributed 
> File System (HDFS), [Amazon S3|https://aws.amazon.com/s3/], Cassandra, 
> MongoDB, and [HBase|https://aws.amazon.com/emr/details/hbase/], and 
> relational data sources such as MySQL, PostgreSQL, [Amazon 
> Redshift|https://aws.amazon.com/redshift/], Microsoft SQL Server, and 
> Teradata.
> This is to track a Ranger plugin for Presto



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 70629: RANGER-2414: Enhancements to support roles in Ranger policies

[Don Bosco Durai]

> On May 11, 2019, 7:10 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java
> > Lines 944 (patched)
> > <https://reviews.apache.org/r/70629/diff/1/?file=2144531#file2144531line944>
> >
> > Do we have small window where the roles could be empty and it could 
> > affect during multi-thread environment>
> 
> Abhay Kulkarni wrote:
> I don't think so. Are you suggesting concurrent updates to policy may 
> lead to inconsistent policy state? If so, one of the transactions will be 
> aborted when attempting to persist changes to database.
> 
> Don Bosco Durai wrote:
> I meant, while the policies are getting updated, a request for 
> authorization, is it possible the  list will be empty?
> 
> Abhay Kulkarni wrote:
> Policies in the policy-engine are treated as read-only during 
> authorization. So, there is no possibility of list getting modified.

Thanks for clarifying.


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70629/#review215198
---


On May 15, 2019, 1:58 a.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70629/
> ---
> 
> (Updated May 15, 2019, 1:58 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin 
> Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-2414
> https://issues.apache.org/jira/browse/RANGER-2414
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Current Ranger policy model supports 
> authorization/column-masking/row-filtering for users/user-groups based on 
> various criteria like accessed-resource, resource-classifications, IP-address 
> and custom conditions. Given the wide-spread use of role-based authorization 
> in traditional enterprise applications (like RDBMS, J2EE), it will be very 
> useful for Ranger policy model to support 'roles' i.e. to be able to specify 
> authorization/column-masking/row-filtering for roles as well - in addition to 
> existing support for users and user-groups.
> 
> This patch provides an initial implementation of support for roles in Ranger.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/errors/ValidationErrorCode.java
>  3111037ff 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 3cf509d7c 
>   agents-common/src/main/java/org/apache/ranger/plugin/model/RangerRole.java 
> PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java
>  990aab0c9 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngine.java
>  9ed500c50 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  eab2c238e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceACLs.java
>  eafbde246 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java
>  a57b39827 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyItemEvaluator.java
>  45231e739 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerOptimizedPolicyEvaluator.java
>  47b4921ad 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerPolicyEvaluator.java
>  5400f71c4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerPolicyItemEvaluator.java
>  a6e24c609 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  5a18226fe 
>   agents-common/src/main/java/org/apache/ranger/plugin/store/RoleStore.java 
> PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerAccessRequestUtil.java
>  c20ccded6 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
> e22249ac6 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  cbd2cb012 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerPolicyValidator.java
>  2c1de4eb8 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyACLs.java
>  e92a2e658 
>   
> agents-comm

Re: Review Request 70629: RANGER-2414: Enhancements to support roles in Ranger policies

[Don Bosco Durai]

> On May 11, 2019, 7:10 a.m., Don Bosco Durai wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java
> > Lines 944 (patched)
> > <https://reviews.apache.org/r/70629/diff/1/?file=2144531#file2144531line944>
> >
> > Do we have small window where the roles could be empty and it could 
> > affect during multi-thread environment>
> 
> Abhay Kulkarni wrote:
> I don't think so. Are you suggesting concurrent updates to policy may 
> lead to inconsistent policy state? If so, one of the transactions will be 
> aborted when attempting to persist changes to database.

I meant, while the policies are getting updated, a request for authorization, 
is it possible the  list will be empty?


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70629/#review215198
---


On May 14, 2019, 1:55 a.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70629/
> ---
> 
> (Updated May 14, 2019, 1:55 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin 
> Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-2414
> https://issues.apache.org/jira/browse/RANGER-2414
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Current Ranger policy model supports 
> authorization/column-masking/row-filtering for users/user-groups based on 
> various criteria like accessed-resource, resource-classifications, IP-address 
> and custom conditions. Given the wide-spread use of role-based authorization 
> in traditional enterprise applications (like RDBMS, J2EE), it will be very 
> useful for Ranger policy model to support 'roles' i.e. to be able to specify 
> authorization/column-masking/row-filtering for roles as well - in addition to 
> existing support for users and user-groups.
> 
> This patch provides an initial implementation of support for roles in Ranger.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/model/AuthzAuditEvent.java 
> 28db58cd9 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java
>  5e2c49211 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/errors/ValidationErrorCode.java
>  3111037ff 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 3cf509d7c 
>   agents-common/src/main/java/org/apache/ranger/plugin/model/RangerRole.java 
> PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java
>  990aab0c9 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngine.java
>  9ed500c50 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  365edcf35 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceACLs.java
>  eafbde246 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java
>  a57b39827 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyItemEvaluator.java
>  45231e739 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerOptimizedPolicyEvaluator.java
>  47b4921ad 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerPolicyEvaluator.java
>  5400f71c4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerPolicyItemEvaluator.java
>  a6e24c609 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  5a18226fe 
>   agents-common/src/main/java/org/apache/ranger/plugin/store/RoleStore.java 
> PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerAccessRequestUtil.java
>  c20ccded6 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
> e22249ac6 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  cbd2cb012 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerPolicyValidator.java
>  2c1de4eb8 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyACLs.java
>  e92a2e658 
>   
> agents-common/src/test/java/or

Re: Review Request 70629: RANGER-2414: Enhancements to support roles in Ranger policies

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70629/#review215198
---




agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java
Lines 944 (patched)
<https://reviews.apache.org/r/70629/#comment301746>

Do we have small window where the roles could be empty and it could affect 
during multi-thread environment>


- Don Bosco Durai


On May 11, 2019, 1:45 a.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70629/
> ---
> 
> (Updated May 11, 2019, 1:45 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin 
> Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-2414
> https://issues.apache.org/jira/browse/RANGER-2414
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Current Ranger policy model supports 
> authorization/column-masking/row-filtering for users/user-groups based on 
> various criteria like accessed-resource, resource-classifications, IP-address 
> and custom conditions. Given the wide-spread use of role-based authorization 
> in traditional enterprise applications (like RDBMS, J2EE), it will be very 
> useful for Ranger policy model to support 'roles' i.e. to be able to specify 
> authorization/column-masking/row-filtering for roles as well - in addition to 
> existing support for users and user-groups.
> 
> This patch provides an initial implementation of support for roles in Ranger.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/model/AuthzAuditEvent.java 
> 28db58cd9 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java
>  5e2c49211 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/errors/ValidationErrorCode.java
>  3111037ff 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
> 3cf509d7c 
>   agents-common/src/main/java/org/apache/ranger/plugin/model/RangerRole.java 
> PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java
>  990aab0c9 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngine.java
>  9ed500c50 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
>  365edcf35 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceACLs.java
>  eafbde246 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java
>  a57b39827 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyItemEvaluator.java
>  45231e739 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerOptimizedPolicyEvaluator.java
>  47b4921ad 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerPolicyEvaluator.java
>  5400f71c4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerPolicyItemEvaluator.java
>  a6e24c609 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  5a18226fe 
>   agents-common/src/main/java/org/apache/ranger/plugin/store/RoleStore.java 
> PRE-CREATION 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerAccessRequestUtil.java
>  c20ccded6 
>   agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
> e22249ac6 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java
>  cbd2cb012 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerPolicyValidator.java
>  2c1de4eb8 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyACLs.java
>  e92a2e658 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyEngine.java
>  5a47ba401 
>   agents-common/src/test/resources/policyengine/test_aclprovider_default.json 
> b4c4def85 
>   
> agents-common/src/test/resources/policyengine/test_policyengine_with_roles.json
>  PRE-CREATION 
>   
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>  f204c15c0 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuditHandler

[jira] [Commented] (RANGER-2410) Build Ranger Using Docker

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827778#comment-16827778
 ] 

Don Bosco Durai commented on RANGER-2410:
-

One time to rebuild your image and output:
./build_ranger_using_docker.sh -build_image mvn clean
 
Subsequently:
#For full build
./build_ranger_using_docker.sh
 
#For selective build
./build_ranger_using_docker.sh  mvn 
 

> Build Ranger Using Docker
> -
>
> Key: RANGER-2410
> URL: https://issues.apache.org/jira/browse/RANGER-2410
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 1.2.0
>Reporter: Helene Treadwell
>Assignee: Don Bosco Durai
>Priority: Minor
> Attachments: build_ranger_using_docker.sh
>
>
> build_ranger_user_docker.sh script throws an error due to dependency on gosu 
> tool
>  
> gpg: directory `/root/.gnupg' created
> gpg: new configuration file `/root/.gnupg/gpg.conf' created
> gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during 
> this run
> gpg: keyring `/root/.gnupg/secring.gpg' created
> gpg: keyring `/root/.gnupg/pubring.gpg' created
> gpg: requesting key BF357DD4 from hkp server pool.sks-keyservers.net
> gpgkeys: HTTP fetch error 7: Failed to connect to ::::c62e:cb61: 
> Cannot assign requested address
> gpg: no valid OpenPGP data found.
> gpg: Total number processed: 0
> The command '/bin/sh -c gpg --keyserver pool.sks-keyservers.net --recv-keys 
> B42F6819007F00F88E364FD4036A9C25BF357DD4 && curl -o /usr/local/bin/gosu 
> -SL "[https://github.com/tianon/gosu/releases/download/1.10/gosu-amd64"
> |https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tianon_gosu_releases_download_1.10_gosu-2Damd64-2522=DwQFaQ=7DfhQjPWzR3PmWBQVpi-kw=kcVpZ83Oz_eaC5ai7r0u5Lr9tm-XLxYP8p3M7dVqMRE=bcXk0KhABoVxKnrS1FIsgI8BKa-U4FvSrHJnzA4SFDg=RYGGYQHlDvfGkdqLPgG44BcFNK5vslmXllryWfj6LIA=]
>  && curl -o /usr/local/bin/gosu.asc -SL 
> "[https://github.com/tianon/gosu/releases/download/1.10/gosu-amd64.asc"
> |https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tianon_gosu_releases_download_1.10_gosu-2Damd64.asc-2522=DwQFaQ=7DfhQjPWzR3PmWBQVpi-kw=kcVpZ83Oz_eaC5ai7r0u5Lr9tm-XLxYP8p3M7dVqMRE=bcXk0KhABoVxKnrS1FIsgI8BKa-U4FvSrHJnzA4SFDg=uZbaTTs55_-ksH4_AXsLglep92TP23tQ4gnu8luy8DE=]
>  && gpg --verify /usr/local/bin/gosu.asc && rm /usr/local/bin/gosu.asc
>  && rm -r /root/.gnupg/ && chmod +x /usr/local/bin/gosu' returned a 
> non-zero code: 2
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2410) Build Ranger Using Docker

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai updated RANGER-2410:

Attachment: build_ranger_using_docker.sh

> Build Ranger Using Docker
> -
>
> Key: RANGER-2410
> URL: https://issues.apache.org/jira/browse/RANGER-2410
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 1.2.0
>Reporter: Helene Treadwell
>Assignee: Don Bosco Durai
>Priority: Minor
> Attachments: build_ranger_using_docker.sh
>
>
> build_ranger_user_docker.sh script throws an error due to dependency on gosu 
> tool
>  
> gpg: directory `/root/.gnupg' created
> gpg: new configuration file `/root/.gnupg/gpg.conf' created
> gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during 
> this run
> gpg: keyring `/root/.gnupg/secring.gpg' created
> gpg: keyring `/root/.gnupg/pubring.gpg' created
> gpg: requesting key BF357DD4 from hkp server pool.sks-keyservers.net
> gpgkeys: HTTP fetch error 7: Failed to connect to ::::c62e:cb61: 
> Cannot assign requested address
> gpg: no valid OpenPGP data found.
> gpg: Total number processed: 0
> The command '/bin/sh -c gpg --keyserver pool.sks-keyservers.net --recv-keys 
> B42F6819007F00F88E364FD4036A9C25BF357DD4 && curl -o /usr/local/bin/gosu 
> -SL "[https://github.com/tianon/gosu/releases/download/1.10/gosu-amd64"
> |https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tianon_gosu_releases_download_1.10_gosu-2Damd64-2522=DwQFaQ=7DfhQjPWzR3PmWBQVpi-kw=kcVpZ83Oz_eaC5ai7r0u5Lr9tm-XLxYP8p3M7dVqMRE=bcXk0KhABoVxKnrS1FIsgI8BKa-U4FvSrHJnzA4SFDg=RYGGYQHlDvfGkdqLPgG44BcFNK5vslmXllryWfj6LIA=]
>  && curl -o /usr/local/bin/gosu.asc -SL 
> "[https://github.com/tianon/gosu/releases/download/1.10/gosu-amd64.asc"
> |https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tianon_gosu_releases_download_1.10_gosu-2Damd64.asc-2522=DwQFaQ=7DfhQjPWzR3PmWBQVpi-kw=kcVpZ83Oz_eaC5ai7r0u5Lr9tm-XLxYP8p3M7dVqMRE=bcXk0KhABoVxKnrS1FIsgI8BKa-U4FvSrHJnzA4SFDg=uZbaTTs55_-ksH4_AXsLglep92TP23tQ4gnu8luy8DE=]
>  && gpg --verify /usr/local/bin/gosu.asc && rm /usr/local/bin/gosu.asc
>  && rm -r /root/.gnupg/ && chmod +x /usr/local/bin/gosu' returned a 
> non-zero code: 2
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (RANGER-2410) Build Ranger Using Docker

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai reassigned RANGER-2410:
---

Assignee: Don Bosco Durai

> Build Ranger Using Docker
> -
>
> Key: RANGER-2410
> URL: https://issues.apache.org/jira/browse/RANGER-2410
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 1.2.0
>Reporter: Helene Treadwell
>Assignee: Don Bosco Durai
>Priority: Minor
>
> build_ranger_user_docker.sh script throws an error due to dependency on gosu 
> tool
>  
> gpg: directory `/root/.gnupg' created
> gpg: new configuration file `/root/.gnupg/gpg.conf' created
> gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during 
> this run
> gpg: keyring `/root/.gnupg/secring.gpg' created
> gpg: keyring `/root/.gnupg/pubring.gpg' created
> gpg: requesting key BF357DD4 from hkp server pool.sks-keyservers.net
> gpgkeys: HTTP fetch error 7: Failed to connect to ::::c62e:cb61: 
> Cannot assign requested address
> gpg: no valid OpenPGP data found.
> gpg: Total number processed: 0
> The command '/bin/sh -c gpg --keyserver pool.sks-keyservers.net --recv-keys 
> B42F6819007F00F88E364FD4036A9C25BF357DD4 && curl -o /usr/local/bin/gosu 
> -SL "[https://github.com/tianon/gosu/releases/download/1.10/gosu-amd64"
> |https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tianon_gosu_releases_download_1.10_gosu-2Damd64-2522=DwQFaQ=7DfhQjPWzR3PmWBQVpi-kw=kcVpZ83Oz_eaC5ai7r0u5Lr9tm-XLxYP8p3M7dVqMRE=bcXk0KhABoVxKnrS1FIsgI8BKa-U4FvSrHJnzA4SFDg=RYGGYQHlDvfGkdqLPgG44BcFNK5vslmXllryWfj6LIA=]
>  && curl -o /usr/local/bin/gosu.asc -SL 
> "[https://github.com/tianon/gosu/releases/download/1.10/gosu-amd64.asc"
> |https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tianon_gosu_releases_download_1.10_gosu-2Damd64.asc-2522=DwQFaQ=7DfhQjPWzR3PmWBQVpi-kw=kcVpZ83Oz_eaC5ai7r0u5Lr9tm-XLxYP8p3M7dVqMRE=bcXk0KhABoVxKnrS1FIsgI8BKa-U4FvSrHJnzA4SFDg=uZbaTTs55_-ksH4_AXsLglep92TP23tQ4gnu8luy8DE=]
>  && gpg --verify /usr/local/bin/gosu.asc && rm /usr/local/bin/gosu.asc
>  && rm -r /root/.gnupg/ && chmod +x /usr/local/bin/gosu' returned a 
> non-zero code: 2
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Draft board report for the period ending 03/31/2019

[Don Bosco Durai]

+1

Selva, thanks for putting this together.

Regards

Bosco


On 4/8/19, 9:24 AM, "Selvamohan Neethiraj"  wrote:

Rangers:

Please review and provide your feedback on Apache board report for Apache 
Ranger for the past 3 months …

Thanks,
Selva-





## Description:
 - The Ranger project is a framework to enable, monitor and manage 
comprehensive data security across the Hadoop platform.

## Issues:
 - There are no issues requiring board attention at this time.

## Activity:
 - Team has been working on fixing issues to make Ranger work with Apache 
Hadoop 3.0+ technology suite for releasing next major Ranger version 2.0.0.
 - Also, team has added following new features in Ranger: Security 
Zone,Incremental Policy Upgrade, expand support for other HSM key storage.

## Health report:
 - As we are closer to the tail-end of the major release (Ranger 2.0.0), 
our mailing list counts seems to be lower than usual

## PMC changes:

 - Currently 19 PMC members.
 - Qiang Zhang was added to the PMC on Fri Feb 08 2019

## Committer base changes:

 - Currently 28 committers.
 - No new committers added in the last 3 months
 - Last committer addition was Bhavik Patel at Thu Nov 01 2018

## Releases:

 - Last release was Apache Ranger 1.2.0 on Wed Oct 03 2018

## Mailing list activity:

 - dev@ranger.apache.org:
- 111 subscribers (up 5 in the last 3 months):
- 498 emails sent to list (795 in previous quarter)

 - u...@ranger.apache.org:
- 188 subscribers (up 6 in the last 3 months):
- 114 emails sent to list (15 in previous quarter)


## JIRA activity:

 - 76 JIRA tickets created in the last 3 months
 - 57 JIRA tickets closed/resolved in the last 3 months

Re: Review Request 70057: RANGER-2324 Bootstrapping Solr in Ranger service start-up

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70057/#review213989
---




embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 134 (patched)
<https://reviews.apache.org/r/70057/#comment300140>

We should get principal only if it is Kerberos mode. Otherwise, we will be 
log warning message on every startup


- Don Bosco Durai


On March 25, 2019, 9:42 a.m., bhavik patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70057/
> ---
> 
> (Updated March 25, 2019, 9:42 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani, 
> Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2324
> https://issues.apache.org/jira/browse/RANGER-2324
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> We are handling the solr bootstrapping in below mentioned manner for Ranger
> 1.) Connection to solr
> 2.) Upload Configuration
> 3.) Create Collection
> 4.) Setting ACL
> 
> 
> Diffs
> -
> 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
>  8d32352 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
>  PRE-CREATION 
>   security-admin/scripts/install.properties fdcee1b 
>   security-admin/scripts/setup.sh bd4bd4c 
>   security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml 
> 686f720 
>   security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 4d4a1de 
>   src/main/assembly/admin-web.xml 0296652 
> 
> 
> Diff: https://reviews.apache.org/r/70057/diff/8/
> 
> 
> Testing
> ---
> 
> Tested Below Scenario on ranger manual start / restart
> 1.) Solr configuration were uploaded successfully
> 2.) Solr collections were created successfully
> 3.) ACL were setup as required.
> 
> 
> Thanks,
> 
> bhavik patel
> 
>

Re: Review Request 70057: RANGER-2324 Bootstrapping Solr in Ranger service start-up

[Don Bosco Durai]

> On March 23, 2019, 4:49 p.m., Don Bosco Durai wrote:
> > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
> > Lines 239 (patched)
> > <https://reviews.apache.org/r/70057/diff/7/?file=2133416#file2133416line239>
> >
> > Any reason we are giving write ot others, but not to group?
> 
> bhavik patel wrote:
> Solr user will be creating properties file and data dir on standalone 
> location.
> Added comment for other developer reference

Not sure I understood this:
pfpSet.add(PosixFilePermission.OTHERS_EXECUTE);

pfpSet.add(PosixFilePermission.OTHERS_READ);

pfpSet.add(PosixFilePermission.OTHERS_WRITE);

Does this mean everyone (other) has read, write and execute permission?


- Don Bosco


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70057/#review213946
---


On March 25, 2019, 9:42 a.m., bhavik patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70057/
> ---
> 
> (Updated March 25, 2019, 9:42 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani, 
> Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2324
> https://issues.apache.org/jira/browse/RANGER-2324
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> We are handling the solr bootstrapping in below mentioned manner for Ranger
> 1.) Connection to solr
> 2.) Upload Configuration
> 3.) Create Collection
> 4.) Setting ACL
> 
> 
> Diffs
> -
> 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
>  8d32352 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
>  PRE-CREATION 
>   security-admin/scripts/install.properties fdcee1b 
>   security-admin/scripts/setup.sh bd4bd4c 
>   security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml 
> 686f720 
>   security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 4d4a1de 
>   src/main/assembly/admin-web.xml 0296652 
> 
> 
> Diff: https://reviews.apache.org/r/70057/diff/8/
> 
> 
> Testing
> ---
> 
> Tested Below Scenario on ranger manual start / restart
> 1.) Solr configuration were uploaded successfully
> 2.) Solr collections were created successfully
> 3.) ACL were setup as required.
> 
> 
> Thanks,
> 
> bhavik patel
> 
>

Re: Review Request 70057: RANGER-2324 Bootstrapping Solr in Ranger service start-up

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70057/#review213946
---




embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
Lines 280 (patched)
<https://reviews.apache.org/r/70057/#comment300052>

We are catching the exception and ignoring it. What happens when BootStrap 
is not started?



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 89 (patched)
<https://reviews.apache.org/r/70057/#comment300055>

Good to have the unit in the variable name itself. E.g. _MS for 
milliseconds or _SEC for seconds.



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 169 (patched)
<https://reviews.apache.org/r/70057/#comment300053>

Why would these calls give exceptions? If it does, then solrFileDir could 
be null, which might not be a good thing



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 175 (patched)
<https://reviews.apache.org/r/70057/#comment300054>

Don't see any difference for the path in cloud and standalone more.



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 239 (patched)
<https://reviews.apache.org/r/70057/#comment300059>

Any reason we are giving write ot others, but not to group?



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 257 (patched)
<https://reviews.apache.org/r/70057/#comment300058>

Can we give enough information? E.g. collection name



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 261 (patched)
<https://reviews.apache.org/r/70057/#comment300060>

Same here, give more information of the context during error



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 377 (patched)
<https://reviews.apache.org/r/70057/#comment300061>

Are we assuming that Solr will be always in Kerberos mode?



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 389 (patched)
<https://reviews.apache.org/r/70057/#comment300062>

connect doesn't throw exception, so we should be checking whether it 
succeeded.



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 511 (patched)
<https://reviews.apache.org/r/70057/#comment300056>

Can we print all parameters? Including solr_config_name, 
max_node_per_shards, etc.



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 599 (patched)
<https://reviews.apache.org/r/70057/#comment300063>

Can we document what we are doing here? Seems we are looping and 
setting/overwriting this variable in some cases



embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 602 (patched)
<https://reviews.apache.org/r/70057/#comment300064>

Might be good to put couple of lines of documentation on what permissions 
we are setting and for what purpose


- Don Bosco Durai


On March 22, 2019, 10:03 a.m., bhavik patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70057/
> ---
> 
> (Updated March 22, 2019, 10:03 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani, 
> Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2324
> https://issues.apache.org/jira/browse/RANGER-2324
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> We are handling the solr bootstrapping in below mentioned manner for Ranger
> 1.) Connection to solr
> 2.) Upload Configuration
> 3.) Create Collection
> 4.) Setting ACL
> 
> 
> Diffs
> -
> 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
>  8d32352 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
>  PRE-CREATION 
>   security-admin/scripts/install.properties fdcee1b 
>   security-admin/scripts/setup.sh bd4bd4c 
>   security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml 
> 686f720 
>   security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 4d4a1de 
>   src/main/assembly/admin-web.xml 0296652 
> 
> 
> Dif

[jira] [Commented] (RANGER-2341) Support for Incremental policy updates to improve performance of ranger-admin and plugins by optimal building of policy-engine

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781211#comment-16781211
 ] 

Don Bosco Durai commented on RANGER-2341:
-

[~abhayk] this would be a good feature. Few questions...

> Cache management in ranger-admin is enhanced to use this table to figure out 
> changes using a previously known version number (provided by module 
> requesting updated policies).
Seems more like more like redo logs in database, which I feel is a good 
approach.

> Backward compatibility is maintained with older plugins by adding another 
> parameter to REST API for downloading policies.
Should we do the other way? New plugins should pass the addition param, so that 
older plugins will work without change?
 
> Policy deltas are disabled by default. 
I feel, we should enable this by default. This is a good feature and let the 
plugins decide whether to use or not.

> Policy delta table is cleared of records older than a week on restart of 
> ranger-admin.
I not sure whether restart should be the trigger, but might be okay for now 
till have an inbuilt scheduler. I assume, we will make the the retention period 
configurable.



> Support for Incremental policy updates to improve performance of ranger-admin 
> and plugins by optimal building of policy-engine
> --
>
> Key: RANGER-2341
> URL: https://issues.apache.org/jira/browse/RANGER-2341
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: master
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: master
>
>
> Requirements:
> Currently, every change to any policy causes rebuilding of policy-engine from 
> scratch. There are several disadvantages:
> 1. Compute time for rebuilding
> 2. Large traffic from ranger-admin to each of the plugins
> 3. Large demand on JVM memory system resulting in frequent garbage collection 
> and pauses of JVM.
> It will be more optimal to communicate only the changes and apply them to 
> existing policy-engine.
> Design notes:
> Policy changes are logged into a new database table.
> Cache management in ranger-admin is enhanced to use this table to figure out 
> changes using a previously known version number (provided by module 
> requesting updated policies).
> Policy engine supports update operation that accepts policy-deltas and 
> returns a new policy engine with deltas applied.
> Resource Trie structures are copied from older policy-engine selectively, and 
> not rebuilt from scratch.
> Backward compatibility is maintained with older plugins by adding another 
> parameter to REST API for downloading policies.
> Ranger admin as well as component plugins may be configured to optionally use 
> policy deltas for its internal policy-engines. Policy deltas are disabled by 
> default. In ranger-admin, policy-deltas are enabled in the ranger-admin by 
> setting configuration variable 'ranger.admin.supports.policy.deltas' to true. 
> In individual plugins, policy-deltas are enabled by setting configuration 
> variable 'ranger.plugin..policy.rest.supports.policy.deltas' to 
> "true".
> Policy delta table is cleared of records older than a week on restart of 
> ranger-admin.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2128) Implement SparkSQL plugin

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775777#comment-16775777
 ] 

Don Bosco Durai commented on RANGER-2128:
-

Sorry, I got pulled into other things There are few feedback, let me 
consolidate them and give it to [~Qin Yao]

> Implement SparkSQL plugin
> -
>
> Key: RANGER-2128
> URL: https://issues.apache.org/jira/browse/RANGER-2128
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Affects Versions: 1.1.0
>Reporter: t oo
>Assignee: Kent Yao
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: support_ranger11.tgz
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Implement SparkSQL plugin



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Question on Ranger policy?

[Don Bosco Durai]

Ramesh, this looks good. 

Thanks

Bosco


On 1/23/19, 12:57 PM, "Ramesh Mani"  wrote:

Bosco,

There were some efforts done on this. Please check this out
https://github.com/jjmeyer0/ranger-client

This might help too.

Thanks,
Ramesh

On 1/23/19, 11:20 AM, "Don Bosco Durai"  wrote:

>I got a similar request from someone else also. I feel, lot of them might
>have already wrote their custom code to manage Ranger policies. We can
>see if someone wants to contribute it back to the community.
>
>Reetika, can you can create a JIRA to track this?
>
>Thanks
>
>Bosco
>
>
>On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:
>
>Hi Reetika Agrawal,
>
>Apache Ranger only has REST APIs for the policy operations.
>
>Thanks,
>Ramesh
>
>
>On 1/19/19, 9:11 PM, "reetika agrawal" 
>wrote:
>
>>Hi,
>>Is there any ranger java client available to get all the ranger
>policies
>>instead of using API calls?
>>
>>-- 
>>Thanks,
>>Reetika Agrawal
>
>
>
>

Re: Question on Ranger policy?

[Don Bosco Durai]

Zsombor

 

This looks good. Do you which jar will have the client library? And does it 
generate the JavaDoc for the Java REST API?

 

Thanks

 

Bosco

 

 

From: "Zs." 
Reply-To: 
Date: Wednesday, January 23, 2019 at 12:40 PM
To: ranger 
Cc: "u...@ranger.apache.org" 
Subject: Re: Question on Ranger policy?

 

There is an Enunciate plugin configured, which generates client libraries 
during build time - unfortunately, it's not published anywhere, you have to 
build Ranger by yourself, and I guess, it's not heavily used.

 

Regards,

 Zsombor

 

On Wed, Jan 23, 2019 at 8:20 PM Don Bosco Durai  wrote:

I got a similar request from someone else also. I feel, lot of them might have 
already wrote their custom code to manage Ranger policies. We can see if 
someone wants to contribute it back to the community.

Reetika, can you can create a JIRA to track this?

Thanks

Bosco


On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:

Hi Reetika Agrawal,

Apache Ranger only has REST APIs for the policy operations.

Thanks,
Ramesh


On 1/19/19, 9:11 PM, "reetika agrawal" 
wrote:

>Hi,
>Is there any ranger java client available to get all the ranger policies
>instead of using API calls?
>
>-- 
>Thanks,
>Reetika Agrawal

Re: Question on Ranger policy?

[Don Bosco Durai]

I got a similar request from someone else also. I feel, lot of them might have 
already wrote their custom code to manage Ranger policies. We can see if 
someone wants to contribute it back to the community.

Reetika, can you can create a JIRA to track this?

Thanks

Bosco


On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:

Hi Reetika Agrawal,

Apache Ranger only has REST APIs for the policy operations.

Thanks,
Ramesh


On 1/19/19, 9:11 PM, "reetika agrawal" 
wrote:

>Hi,
>Is there any ranger java client available to get all the ranger policies
>instead of using API calls?
>
>-- 
>Thanks,
>Reetika Agrawal

[jira] [Commented] (RANGER-2324) Bootstrapping Solr in Ranger service start-up

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750238#comment-16750238
 ] 

Don Bosco Durai commented on RANGER-2324:
-

[~bpatel] can we have some more detail on what you are planning to do here? 
Thanks

> Bootstrapping Solr in Ranger service start-up
> -
>
> Key: RANGER-2324
> URL: https://issues.apache.org/jira/browse/RANGER-2324
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: bhavik patel
>Assignee: bhavik patel
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Password encryption for service definitions

[Don Bosco Durai]

I also feel we shouldn't store the key along with the encrypted data. It 
defeats the purpose.

Adam, getting from properties is one way to do it. We have to ensure that the 
key is auto generated per install and we have to ensure that it works in HA 
environment.

Ideally, even the config file shouldn't have the actual key. It should be in 
the keystore with only user Privacera have the read permission. The key to the 
keystore can be in the config file.

The other way is to use Ranger KMS in the future.

Bosco


On 1/8/19, 6:13 AM, "Rempter, A. (Adam)"  wrote:

Hey Zsombor,

Thanks for update. 

I understand that... but it means that (when storing encryption key next to 
password) it is effectively not encrypted. 

According to owasp 
(https://www.owasp.org/index.php/Cryptographic_Storage_Cheat_Sheet#Rule_-_Ensure_that_any_secret_key_is_protected_from_unauthorized_access)

Rule - Store unencrypted keys away from the encrypted data
If the keys are stored with the data then any compromise of the data will 
easily compromise the keys as well. Unencrypted keys should never reside on the 
same machine or cluster as the data.

One solution would be simply not store encryption key in db. It is anyway 
available via configuration key:

public static final String ENCRYPT_KEY = 
PropertiesUtil.getProperty("ranger.password.encryption.key", 
PasswordUtils.DEFAULT_ENCRYPT_KEY)

What do you think?

Thanks,
Adam

-Wiadomość oryginalna-
Od: Zs. [mailto:gzsom...@gmail.com] 
Wysłano: 8 stycznia 2019 14:56
Do: ranger 
Temat: Re: PD: Password encryption for service definitions

Hi,

 The problem is that Ranger needs to know the password, to reach out to the 
service, so it must store the password somewhere in a decryptable state.
Ideally, every service/protocol should support kerberos, so authentication 
could work without passwords.

Regards,
 Zsombor

On Tue, Jan 8, 2019 at 2:21 PM Rempter, A. (Adam) 
 wrote:

> Hello there,
>
> While using Ranger I noticed that when I create service def with input
> property:
>{
> "itemId": 3,
> "name": "password",
> "type": "password",
> "subType": "",
> "mandatory": true,
> "validationRegEx": "",
> "validationMessage": "",
> "uiHint":"",
> "label": "Secret key"
>   }
>
> Ranger will encrypt it using:
>
> if (StringUtils.equalsIgnoreCase(configKey, CONFIG_KEY_PASSWORD)) {
>  String cryptConfigString = CRYPT_ALGO + ","
> +  ENCRYPT_KEY + "," + SALT + "," + ITERATION_COUNT + "," + 
> + configValue;
>  String encryptedPwd = 
> PasswordUtils.encryptPassword(cryptConfigString);
>
> Problem is that all encryption parameters are stored next to password 
> (encryption key and salt):
>
> | 609 | NULL | 2019-01-08 10:07:33 | 2019-01-08 10:07:34 |   1 |
>1 |  82 | password  |
> PBEWithMD5AndDES,tzL1AKl5uc4NKYaoQ4P3WLGIBFPXWPWdu1fRm9004jtQiV,f77aLY
> Lo,1000,6IxJOOpoFsJXyLNjNf/M9Q==
>
> Even if I change default ones in
> $ranger_home/ews/webapp/WEB-INF/classes/conf/ranger-admin-default-site
> .xml,
> they will still be storred in db
>
> Is this know issue? Basically it means that password can be decrypted 
> with little effort…
>
> Thanks,
> Adam Rempter
>
>
> ING Business Shared Services B.V. z siedzibą w Amsterdamie, Holandia, 
> VAT PL 526-319-58-54, działająca w Polsce w formie oddziału, pod firmą 
> ING Business Shared Services B.V. spółka z ograniczoną 
> odpowiedzialnością Oddział w Polsce z siedzibą w Katowicach, ul. 
> Konduktorska 35, 40-155 Katowice, NIP: 2050005130, wpisana do rejestru 
> przedsiębiorców Krajowego Rejestru Sądowego prowadzonego przez Sąd 
> Rejonowy Katowice-Wschód w Katowicach, VIII Wydział Gospodarczy 
> Krajowego Rejestru Sądowego pod numerem KRS 702305.
>

ING Business Shared Services B.V. z siedzibą w Amsterdamie, Holandia, VAT 
PL 526-319-58-54, działająca w Polsce w formie oddziału, pod firmą ING Business 
Shared Services B.V. spółka z ograniczoną odpowiedzialnością Oddział w Polsce z 
siedzibą w Katowicach, ul. Konduktorska 35, 40-155 Katowice, NIP: 2050005130, 
wpisana do rejestru przedsiębiorców Krajowego Rejestru Sądowego prowadzonego 
przez Sąd Rejonowy Katowice-Wschód w Katowicach, VIII Wydział Gospodarczy 
Krajowego Rejestru Sądowego pod numerem KRS 702305.

Re: Subject: [NOTICE] Mandatory relocation of Apache git repositories on git-wip-us.apache.org

[Don Bosco Durai]

Selva, thanks.

Vel, do we need to do any changes on Jenkins or any other places?

Thanks

Bosco


On 12/28/18, 10:12 AM, "Selvamohan Neethiraj"  wrote:

All:

I have created RANGER-2316 
 for updating repo URL in 
Quick Startup Guide and fixed it in the repo.
After update, I have followed the process mentioned in docs/README.txt to 
deploy the changes to documentation site, 
http://ranger.apache.org/quick_start_guide.html


Thanks,
Selva-

> On Dec 17, 2018, at 3:27 PM, Velmurugan Periasamy  wrote:
> 
> Migration has been completed. 
> 
> New target URL(s): 
> - https://gitbox.apache.org/repos/asf/ranger.git 
> - https://github.com/apache/ranger 
> 
> 
> 
>> On Dec 17, 2018, at 2:47 PM, Velmurugan Periasamy  
wrote:
>> 
>> Hello Rangers:
>> 
>> Thank you for your responses. Since there is agreement in the dev list 
(ref: 
https://lists.apache.org/thread.html/fb07264b623c458389c8487e53d0a236341cb733a2055e970e13e1e1@%3Cdev.ranger.apache.org%3E
 
),
 I will go ahead and open an Infra ticket to schedule the relocation of git 
repos.
>> 
>> Thank you,
>> Vel
>> 
>> 
>> From: Gautam Borad mailto:gau...@apache.org>>
>> Sent: Wednesday, December 12, 2018 3:49 AM
>> To: dev@ranger.apache.org 
>> Subject: Re: Subject: [NOTICE] Mandatory relocation of Apache git 
repositories on git-wip-us.apache.org 
>> 
>> +1
>> 
>> On Wed, Dec 12, 2018 at 2:45 AM Zs. mailto:gzsom...@gmail.com>> wrote:
>> 
>>> +1
>>> 
>>> Regards,
>>> Zsombor
>>> 
>>> On Tue, Dec 11, 2018 at 7:28 PM Sailaja Polavarapu <
>>> spolavar...@hortonworks.com > wrote:
>>> 
 +1
 
 On 12/10/18, 1:22 PM, "Abhay Kulkarni" mailto:akulka...@hortonworks.com>>
>>> wrote:
 
+1
 
On 12/10/18, 12:06 PM, "Velmurugan Periasamy" mailto:v...@apache.org>>
>>> wrote:
 
> Rangers:
> 
> Please see below. I propose to move ranger repos over to gitbox
>>> (from
> git-wip-us.apache.org  
>) during the
 first
> phase (voluntary coordination). Please share your thoughts.
> 
> Once there is agreement in dev list, I can open INFRA ticket to
 complete
> the move. Tentative target next week (week of Dec 17).
> 
> Here¹s my +1.
> 
> Thank you,
> Vel
> 
> 
> From: Daniel Gruno mailto:humbed...@apache.org>>
> Sent: Friday, December 7, 2018 11:52 AM
> To: us...@infra.apache.org 
> Subject: [NOTICE] Mandatory relocation of Apache git repositories on
> git-wip-us.apache.org 
> 
> [IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
> DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]
> 
> Hello Apache projects,
> 
> I am writing to you because you may have git repositories on the
> git-wip-us server, which is slated to be decommissioned in the
>>> coming
> months. All repositories will be moved to the new gitbox service
>>> which
> includes direct write access on github as well as the standard ASF
> commit access via gitbox.apache.org .
> 
> ## Why this move? ##
> The move comes as a result of retiring the git-wip service, as the
> hardware it runs on is longing for retirement. In lieu of this, we
> have decided to consolidate the two services (git-wip and gitbox),
>>> to
> ease the management of our repository systems and future-proof the
> underlying hardware. The move is fully automated, and ideally,
>>> nothing
> will change in your workflow other than added features and access to
> GitHub.
> 
> ## Timeframe for relocation ##
> Initially, we are asking that projects voluntarily request to move
> their repositories to gitbox, hence this email. The voluntary
> timeframe is between now and January 9th 2019, during which projects
> are free to either move over to gitbox or stay put on git-wip. After
> this phase, we will be requiring the remaining projects to move
>>> within
> one month, after which we will move the remaining projects over.
> 

[jira] [Commented] (RANGER-2312) Internal users should have permission to modify their personal information in User Profile page.

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728573#comment-16728573
 ] 

Don Bosco Durai commented on RANGER-2312:
-

Hi [~zhangqiang2], I don't have any specific concerns for this change request. 
I was just curious how you were planning to use Name and email. 

I have given my feedback on review board for your patch. Please review

> Internal users should have permission to modify their personal information in 
> User Profile page.
> 
>
> Key: RANGER-2312
> URL: https://issues.apache.org/jira/browse/RANGER-2312
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Minor
>  Labels: patch
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-2312-Internal-users-should-have-permission-to.patch
>
>
> Auditor role users cannot modify their personal user profile.
> User role and KMSAuditor role users have the same problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69626: RANGER-2312 Internal users should have permission to modify their personal information in User Profile page.

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69626/#review211530
---




security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java
Line 211 (original), 211 (patched)
<https://reviews.apache.org/r/69626/#comment296828>

Please verify. It seems, users would be able to provide higher role to 
himself/herself.


- Don Bosco Durai


On Dec. 22, 2018, 3:59 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69626/
> ---
> 
> (Updated Dec. 22, 2018, 3:59 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nitin Galave, pengjianhua, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, sam  rome, Venkat Ranganathan, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-2312
> https://issues.apache.org/jira/browse/RANGER-2312
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Auditor role users cannot modify their personal user profile.
> User role and KMSAuditor role users have the same problem.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java 9e457826e 
>   security-admin/src/main/webapp/scripts/views/user/UserProfileForm.js 
> 5ebd29048 
>   security-admin/src/test/java/org/apache/ranger/biz/TestUserMgr.java 
> 202a113d8 
> 
> 
> Diff: https://reviews.apache.org/r/69626/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

[jira] [Commented] (RANGER-2312) Users should have permission to modify their personal information in User Profile page.

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727949#comment-16727949
 ] 

Don Bosco Durai commented on RANGER-2312:
-

Hi [~zhangqiang2]

There were couple of changes from your side to User management. The users in 
Ranger are not the real source of truth. It is just there to make it easy to 
create policies and also login into Ranger. The real source is AD/LDAP. 
Ideally, the changes done in AD/LDAP should be sync'ed automatically into 
Ranger.

Are you envisioning that these users or their details will be used in other 
ways by Ranger or dependent components? 

Thanks

> Users should have permission to modify their personal information in User 
> Profile page.
> ---
>
> Key: RANGER-2312
> URL: https://issues.apache.org/jira/browse/RANGER-2312
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Minor
>  Labels: patch
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-2312-Users-should-have-permission-to-modify-t.patch
>
>
> Auditor role users cannot modify their personal user profile.
> User role and KMSAuditor role users have the same problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Subject: [NOTICE] Mandatory relocation of Apache git repositories on git-wip-us.apache.org

[Don Bosco Durai]

+1 from my side.

Thanks

Bosco


On 12/10/18, 12:06 PM, "Velmurugan Periasamy"  wrote:

Rangers:

Please see below. I propose to move ranger repos over to gitbox (from 
git-wip-us.apache.org ) during the first phase 
(voluntary coordination). Please share your thoughts. 

Once there is agreement in dev list, I can open INFRA ticket to complete 
the move. Tentative target next week (week of Dec 17). 

Here’s my +1. 

Thank you,
Vel


From: Daniel Gruno 
Sent: Friday, December 7, 2018 11:52 AM
To: us...@infra.apache.org
Subject: [NOTICE] Mandatory relocation of Apache git repositories on 
git-wip-us.apache.org

[IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
  DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]

Hello Apache projects,

I am writing to you because you may have git repositories on the
git-wip-us server, which is slated to be decommissioned in the coming
months. All repositories will be moved to the new gitbox service which
includes direct write access on github as well as the standard ASF
commit access via gitbox.apache.org.

## Why this move? ##
The move comes as a result of retiring the git-wip service, as the
hardware it runs on is longing for retirement. In lieu of this, we
have decided to consolidate the two services (git-wip and gitbox), to
ease the management of our repository systems and future-proof the
underlying hardware. The move is fully automated, and ideally, nothing
will change in your workflow other than added features and access to
GitHub.

## Timeframe for relocation ##
Initially, we are asking that projects voluntarily request to move
their repositories to gitbox, hence this email. The voluntary
timeframe is between now and January 9th 2019, during which projects
are free to either move over to gitbox or stay put on git-wip. After
this phase, we will be requiring the remaining projects to move within
one month, after which we will move the remaining projects over.

To have your project moved in this initial phase, you will need:

- Consensus in the project (documented via the mailing list)
- File a JIRA ticket with INFRA to voluntarily move your project repos
   over to gitbox (as stated, this is highly automated and will take
   between a minute and an hour, depending on the size and number of
   your repositories)

To sum up the preliminary timeline;

- December 9th 2018 -> January 9th 2019: Voluntary (coordinated)
   relocation
- January 9th -> February 6th: Mandated (coordinated) relocation
- February 7th: All remaining repositories are mass migrated.

This timeline may change to accommodate various scenarios.

## Using GitHub with ASF repositories ##
When your project has moved, you are free to use either the ASF
repository system (gitbox.apache.org) OR GitHub for your development
and code pushes. To be able to use GitHub, please follow the primer
at: https://reference.apache.org/committer/github


We appreciate your understanding of this issue, and hope that your
project can coordinate voluntarily moving your repositories in a
timely manner.

All settings, such as commit mail targets, issue linking, PR
notification schemes etc will automatically be migrated to gitbox as
well.

With regards, Daniel on behalf of ASF Infra.

PS:For inquiries, please reply to us...@infra.apache.org, not your
project's dev list :-).

Re: Allow clients to supply tag information

[Don Bosco Durai]

Hi Bolke

Thanks for the suggestion and contribution.

I am trying to understand your approach. Can you add your patch in Review 
Board. It will be easy to see the changes you have done visually.

I have a few questions and design suggestions.
1. When you say "Client", are you referring to "Plugin Implementation Code" or 
"End User" (similar to Accumulo model)
2. If you meant "Plugin" or "Custom Plugin", then I feel it is a good 
suggestion and we should support it. It is end-user, then it is a longer 
discussion
3. Based on the discussion so far and reviewing the code change at high level, 
it seems you are extending at the Tag Enricher level. Alternatively, would it 
me more design friendly to provide a method/API in the plugin interface to 
return or override the tags. E.g.  getTags( request ). Custom plugins can 
override this method and alter the Tags to be returned. This might be more 
isolated and cleaner implementation, so the Plugin writers can only focus on 
their Plugin implementation.
4. If needed, for advanced users, we can provide an interface or API to 
implement their own Tag Sync. Which could be in addition to Atlas/Kafka or 
exclusive to their environment or Meta Store.

Thanks

Bosco



On 12/5/18, 11:59 AM, "Bolke de Bruin"  wrote:

Hi Abhay,

Also answers inline.

B.

Verstuurd vanaf mijn iPad

> Op 5 dec. 2018 om 20:25 heeft Abhay Kulkarni  
het volgende geschreven:
> 
> Hi Bolke,
> 
> My comments inline.
> 
> Thanks,
> -Abhay
> 
>> On 12/4/18, 1:07 PM, "Bolke de Bruin"  wrote:
>> 
>> Hi Abhay,
>> 
>> Good point on #1 will take that into account if possible (can a enricher
>> call audit events?).
>> 
>> On #2 yes, otherwise the resource matcher will stop working. Maybe proper
>> namespacing is the way to go here. Implementing it this way ensures
>> backwards compatibility. On a broader thought, I think Ranger is lacking
>> here. Context could also be provided by the client and there is no real
>> clean way of doing this at the moment.
> 
> Abhay> I will need to take a look to figure out why resource matcher will
> not work. However, instead of implementing a new API (removeValue()), is
> it possible to use setValue() API to set KEY_CLIENT_TAG entry to null?

I don’t think that is possible. The resource matcher checks for elements 
and setting it to null means it is present which means the signature still 
doesn’t match.

>> 
>> Question should client tags only apply to SELF, or also
>> SELF_OR_DESCENDENT and ANCESTOR? I wasn’t sure here.
> 
> Abhay> I don’t see any issue, at this time, to apply client-tags when
> match-type is SELF, SELF_OR_DESCENDENT or ANCESTOR.

This means a client tag will match against all of them at any time. The 
client isn’t aware of match-types. Correct?

>> 
>> Second question (a bit unrelated): how scaleable is the tagsync approach?
>> If we have millions of tagged files and sources they all end up being
>> registered in Ranger this could easily grow exponentially. Besides
>> getting outdated? The other approach could be to have this handled in the
>> client (pickup info from TagSource - ie. Atlas and supply this to the
>> policy engine).
> 
> Abhay> I see that there is some lag involved. But, overall, the
> architecture allows for tag-based policies (really ABAC way of
> authorization) to be applied across all components uniformly. Having
> ranger-admin as a central repository of policies and tags, and components
> as simply clients downloading these artifacts has many more advantages
> than each component having to do all the work by itself. Also, any Kafka
> delay will also be an issue even when components directly received tags
> from Atlas without ranger-admin mediating tag transfer. Moreover, there
> are several optimizations possible (such as incremental download of tags -
> not implemented yet) which can speed up tag downloads significantly. With
> a large number of tags, surely, the size of ranger-admin tag tables will
> increase, but IMO, it is a fair trade-off considering all other advantages
> this architecture provides us. Also, it will be useful to know the order
> of magnitude of delay you experienced (other than possibly up to 1 minute
> delay because of the interval between tag downloads).

The one minute is already too much for us. The example I gave happens 
within a few milliseconds so basically any delay is not acceptable.

To me it seems architecturally incorrect to have Ranger to be a source for 
tags as that is  Atlas (or some other). Ranger is duplicating things here 
rather than sticking to what it is good at: policies.  Clients are already 
downloading tags, doing that from Atlas instead of Ranger is not adding a lot 
of complexity and can be 

[jira] [Closed] (RANGER-2284) Unable to build image using docker

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai closed RANGER-2284.
---

> Unable to build image using docker
> --
>
> Key: RANGER-2284
> URL: https://issues.apache.org/jira/browse/RANGER-2284
> Project: Ranger
>  Issue Type: Bug
>  Components: build-infra
>Affects Versions: master
>Reporter: Nelson Costa
>Assignee: Don Bosco Durai
>Priority: Major
> Attachments: build_ranger_using_docker.sh
>
>
> Running `./build_ranger_using_docker.sh -build_image` fails with error:
> {noformat}
>  ---> 62b699d731cb
> Step 11/27 : ADD 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  /tools
> ADD failed: failed to GET 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  with status 404 Not Found: 
> 
> 404 Not Found
> 
> Not Found
> The requested URL 
> /dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1 was not 
> found on this server.
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69370: RANGER-2284: Unable to build image using docker

[Don Bosco Durai]

Hi Rangers…

 

Can one of you help me review this request and provide a “ship it” if it is 
okay? Nelson Costa has already verified it works 
https://issues.apache.org/jira/browse/RANGER-2284

 

Thanks

 

Bosco

 

 

From: Don Bosco Durai  on behalf of Don Bosco Durai 

Reply-To: Don Bosco Durai 
Date: Thursday, November 15, 2018 at 10:31 PM
To: Don Bosco Durai , ranger 
Subject: Review Request 69370: RANGER-2284: Unable to build image using docker

 

This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/69370/ 

 

Review request for ranger.
By Don Bosco Durai.
Bugs: RANGER-2284 
Repository: ranger 
Description 
Older maven version link had expired. Updated maven version
Testing 
Tested on Ubuntu server
Diffs 
build_ranger_using_docker.sh (82b7179da)
View Diff

 

[jira] [Resolved] (RANGER-542) Remove MPL license text from license file

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai resolved RANGER-542.

Resolution: Fixed

Fixed as part of other changes

> Remove MPL license text from license file
> -
>
> Key: RANGER-542
> URL: https://issues.apache.org/jira/browse/RANGER-542
> Project: Ranger
>  Issue Type: Bug
>Affects Versions: 0.5.0
>    Reporter: Don Bosco Durai
>Priority: Critical
>
> We need to remove MPL license text from license file. 
> Refer to JIRA https://issues.apache.org/jira/browse/RANGER-316 and release 
> feedback http://s.apache.org/NBH



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (RANGER-542) Remove MPL license text from license file

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai closed RANGER-542.
--

> Remove MPL license text from license file
> -
>
> Key: RANGER-542
> URL: https://issues.apache.org/jira/browse/RANGER-542
> Project: Ranger
>  Issue Type: Bug
>Affects Versions: 0.5.0
>    Reporter: Don Bosco Durai
>Priority: Critical
>
> We need to remove MPL license text from license file. 
> Refer to JIRA https://issues.apache.org/jira/browse/RANGER-316 and release 
> feedback http://s.apache.org/NBH



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-1986) Hive Policies doesn't access columns with spaces

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690714#comment-16690714
 ] 

Don Bosco Durai commented on RANGER-1986:
-

[~nitin.galave] sorry for the delay. I feel we should have consistent behavior 
for all resources. Even HDFS files could have spaces in them. If we all spaces, 
we should trim white spaces before saving to DB. 

> Hive Policies doesn't access columns with spaces
> 
>
> Key: RANGER-1986
> URL: https://issues.apache.org/jira/browse/RANGER-1986
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>    Reporter: Don Bosco Durai
>Assignee: Nitin Galave
>Priority: Major
>
> Hive supports columns with spaces, but Ranger doesn't. When space is given 
> while entering the column name in the Ranger Policy UI, it automatically 
> splits it into 2 words/column names.
> Is it possible to make "enter" has the way to accept/finalize the column 
> name? Or if a single or double quote is given, then wait for another single 
> or double quotes or enter before accepting the column name?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 69370: RANGER-2284: Unable to build image using docker

[Don Bosco Durai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69370/
---

Review request for ranger.


Bugs: RANGER-2284
https://issues.apache.org/jira/browse/RANGER-2284


Repository: ranger


Description
---

Older maven version link had expired. Updated maven version


Diffs
-

  build_ranger_using_docker.sh 82b7179da 


Diff: https://reviews.apache.org/r/69370/diff/1/


Testing
---

Tested on Ubuntu server


Thanks,

Don Bosco Durai

[jira] [Commented] (RANGER-2284) Unable to build image using docker

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687322#comment-16687322
 ] 

Don Bosco Durai commented on RANGER-2284:
-

[~nelsonc] can you try the attached script? If it works for you, I can commit.

Thanks

> Unable to build image using docker
> --
>
> Key: RANGER-2284
> URL: https://issues.apache.org/jira/browse/RANGER-2284
> Project: Ranger
>  Issue Type: Bug
>  Components: build-infra
>Affects Versions: master
>Reporter: Nelson Costa
>Assignee: Don Bosco Durai
>Priority: Major
> Attachments: build_ranger_using_docker.sh
>
>
> Running `./build_ranger_using_docker.sh -build_image` fails with error:
> {noformat}
>  ---> 62b699d731cb
> Step 11/27 : ADD 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  /tools
> ADD failed: failed to GET 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  with status 404 Not Found: 
> 
> 404 Not Found
> 
> Not Found
> The requested URL 
> /dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1 was not 
> found on this server.
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2284) Unable to build image using docker

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai updated RANGER-2284:

Attachment: build_ranger_using_docker.sh

> Unable to build image using docker
> --
>
> Key: RANGER-2284
> URL: https://issues.apache.org/jira/browse/RANGER-2284
> Project: Ranger
>  Issue Type: Bug
>  Components: build-infra
>Affects Versions: master
>Reporter: Nelson Costa
>Assignee: Don Bosco Durai
>Priority: Major
> Attachments: build_ranger_using_docker.sh
>
>
> Running `./build_ranger_using_docker.sh -build_image` fails with error:
> {noformat}
>  ---> 62b699d731cb
> Step 11/27 : ADD 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  /tools
> ADD failed: failed to GET 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  with status 404 Not Found: 
> 
> 404 Not Found
> 
> Not Found
> The requested URL 
> /dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1 was not 
> found on this server.
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (RANGER-2284) Unable to build image using docker

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai reassigned RANGER-2284:
---

Assignee: Don Bosco Durai

> Unable to build image using docker
> --
>
> Key: RANGER-2284
> URL: https://issues.apache.org/jira/browse/RANGER-2284
> Project: Ranger
>  Issue Type: Bug
>  Components: build-infra
>Affects Versions: master
>Reporter: Nelson Costa
>Assignee: Don Bosco Durai
>Priority: Major
>
> Running `./build_ranger_using_docker.sh -build_image` fails with error:
> {noformat}
>  ---> 62b699d731cb
> Step 11/27 : ADD 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  /tools
> ADD failed: failed to GET 
> https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1
>  with status 404 Not Found: 
> 
> 404 Not Found
> 
> Not Found
> The requested URL 
> /dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1 was not 
> found on this server.
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2232) Security Zones feature in Apache Ranger

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683145#comment-16683145
 ] 

Don Bosco Durai commented on RANGER-2232:
-

[~abhayk] This is a very good feature and you have articulated it very well in 
the document.

+1 from my side.

I do have a suggestion, while we are implementing this feature, let's not make 
it very much tied to resource policies. We should be able to use security zones 
for other purposes also. E.g. In the future, we should be able to implement 
other features like https://issues.apache.org/jira/browse/RANGER-693 using 
security zones.


> Security Zones feature in Apache Ranger
> ---
>
> Key: RANGER-2232
> URL: https://issues.apache.org/jira/browse/RANGER-2232
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin
>Reporter: Madhan Neethiraj
>Assignee: Abhay Kulkarni
>Priority: Major
> Attachments: Apache Ranger - Security Zones.pdf
>
>
> This is to introduce a new abstraction in Apache Ranger that would allow 
> carving/bucketing of resources in a service into multiple zones, for better 
> administration of security policies. This would enable multiple 
> administrators to setup security policies for a service – based on the zones 
> to which they have been granted administration rights. 
> For example, let us consider 2 security zones ‘finance’ and ‘sales’:
>  - Security zone ‘finance’ includes all contents in Hive database named 
> ‘finance’ 
>  - Security zone ‘sales’ includes all contents in ‘sales’ database 
>  - Set of users and groups are designated as administrators each zone 
>  - Users are allowed to setup policies only in zones in which they are 
> administrators 
>  - Policies defined in a zone are applicable only for resources of the zone
>  - A zone can be extended to include resource from multiple services like 
> HDFS, Hive, HBase, Kafka, .., allowing administrators of a zone to setup 
> policies for resources owned by their organization across multiple services.
>  - Audit logs will include name of the zone in which the accessed resource 
> resides. Only users having appropriate permissions on the security zone can 
> view its audit logs.
> Attached document has more details on various aspects of Security Zones.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2128) Implement SparkSQL plugin

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682739#comment-16682739
 ] 

Don Bosco Durai commented on RANGER-2128:
-

[~Qin Yao] thanks. Let me coordinate with you on your changes. Since I was 
blocked, I ended taking out all references to Hive Context, which I feel is a 
cleaner option. We should plan to merge both our codes.

> Implement SparkSQL plugin
> -
>
> Key: RANGER-2128
> URL: https://issues.apache.org/jira/browse/RANGER-2128
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Affects Versions: 1.1.0
>Reporter: t oo
>Assignee: Kent Yao
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: support_ranger11.tgz
>
>
> Implement SparkSQL plugin



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2281) Support Trusted Proxy in ranger

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682086#comment-16682086
 ] 

Don Bosco Durai commented on RANGER-2281:
-

[~spolavarapu] can you look into this JIRA also? 
https://issues.apache.org/jira/browse/RANGER-2049

I had created this with the intention that RangerAdmin will support doAs. If 
you feel they are the same, then we can close one of them.

Thanks

> Support Trusted Proxy in ranger
> ---
>
> Key: RANGER-2281
> URL: https://issues.apache.org/jira/browse/RANGER-2281
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.0.0
>Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 2.0.0
>
>
> Ranger kerberos authentication module should support the notion of 
> proxy-user, who would be allowed to perform operations on behalf of other 
> users i.e. impersonate other users - similar to Hadoop.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ranger mvn build parameters...

[Don Bosco Durai]

There are 2 patches. Any specific one you want me to try it out?

Bosco


On 10/25/18, 5:18 PM, "Sailaja Polavarapu"  wrote:

Hi, Bosco,
 I updated a new patch to 
https://issues.apache.org/jira/browse/RANGER-2265. Can you give it a try? 

On 10/25/18, 2:22 PM, "Don Bosco Durai"  wrote:

Yes, I am using Linux based Docker.

Thanks for looking into it.

Bosco


On 10/25/18, 2:09 PM, "Sailaja Polavarapu" 
 wrote:

From the following documentation it looks like profile with OS 
family takes precedence.


https://maven.apache.org/guides/introduction/introduction-to-profiles.html

>>>>>>>>>>>>>>>>>>

Profiles can also be active by default using a configuration like 
the following:





  

profile-1



  true



...

  



This profile will automatically be active for all builds unless 
another profile in the same POM is activated using one of the previously 
described methods. All profiles that are active by default are automatically 
deactivated when a profile in the POM is activated on the command line or 
through its activation config.

>>>>>>>>>>>>>>>>>>>>>>>>



If you are compiling on linux, then you see this issue as the 
profile with os family is taking precedence. If you are compiling in mac, we 
don’t see this issue. One more solution that is specified in the above document 
is to use settings.xml to specify active profiles.



Thanks,

Sailaja.



On 10/25/18, 11:21 AM, "Velmurugan Periasamy"  
wrote:



I am able to build locally using the below command (no change) 
even after RANGER-2243.



mvn clean compile package install assembly:assembly



Could it be related to maven version?  I am using maven 3.3.9 
and it is picking up the right maven-assembly-plugin (2.2-beta-5) specified in 
pom.xml. Not sure why you are picking up 3.0.0?





From: Ramesh Mani 

Sent: Thursday, October 25, 2018 2:15 PM

To: dev@ranger.apache.org

Subject: Re: Ranger mvn build parameters...



Hi Bosco,



JIRA https://issues.apache.org/jira/browse/RANGER-2265 is 
raised for

addressing this. If this patch works we don¹t  need to updated
    
    

Thanks

Ramesh



On 10/23/18, 10:22 PM, "Don Bosco Durai"  
wrote:



>Seems after https://issues.apache.org/jira/browse/RANGER-2243 
patch, we

>need to pass -Pall to the build.

>

>

>

>Without that, I am getting this error:

>

>

>

>mvn -DskipTests=true clean compile package install 
assembly:assembly

>

>[ERROR] Could not find goal 'assembly' in plugin

>org.apache.maven.plugins:maven-assembly-plugin:3.0.0 among 
available

>goals help, single -> [Help 1]

>

>

>

>Can we change the public documentation with additional 
details? Seems we

>need to change quite a few places:

>

>

>

>https://ranger.apache.org/quick_start_guide.html

  

Re: Ranger mvn build parameters...

[Don Bosco Durai]

Yes, I am using Linux based Docker.

Thanks for looking into it.

Bosco


On 10/25/18, 2:09 PM, "Sailaja Polavarapu"  wrote:

From the following documentation it looks like profile with OS family takes 
precedence.

https://maven.apache.org/guides/introduction/introduction-to-profiles.html

>>>>>>>>>>>>>>>>>>

Profiles can also be active by default using a configuration like the 
following:





  

profile-1



  true



...

  



This profile will automatically be active for all builds unless another 
profile in the same POM is activated using one of the previously described 
methods. All profiles that are active by default are automatically deactivated 
when a profile in the POM is activated on the command line or through its 
activation config.

>>>>>>>>>>>>>>>>>>>>>>>>



If you are compiling on linux, then you see this issue as the profile with 
os family is taking precedence. If you are compiling in mac, we don’t see this 
issue. One more solution that is specified in the above document is to use 
settings.xml to specify active profiles.



Thanks,

Sailaja.



On 10/25/18, 11:21 AM, "Velmurugan Periasamy"  wrote:



I am able to build locally using the below command (no change) even 
after RANGER-2243.



mvn clean compile package install assembly:assembly



Could it be related to maven version?  I am using maven 3.3.9 and it is 
picking up the right maven-assembly-plugin (2.2-beta-5) specified in pom.xml. 
Not sure why you are picking up 3.0.0?





From: Ramesh Mani 

Sent: Thursday, October 25, 2018 2:15 PM

To: dev@ranger.apache.org

Subject: Re: Ranger mvn build parameters...



Hi Bosco,



JIRA https://issues.apache.org/jira/browse/RANGER-2265 is raised for

addressing this. If this patch works we don¹t  need to updated



Thanks

Ramesh



On 10/23/18, 10:22 PM, "Don Bosco Durai"  wrote:



>Seems after https://issues.apache.org/jira/browse/RANGER-2243 patch, we

>need to pass -Pall to the build.

>

>

>

>Without that, I am getting this error:

>

>

>

>mvn -DskipTests=true clean compile package install assembly:assembly

>

>[ERROR] Could not find goal 'assembly' in plugin

>org.apache.maven.plugins:maven-assembly-plugin:3.0.0 among available

>goals help, single -> [Help 1]

>

>

>

>Can we change the public documentation with additional details? Seems 
we

>need to change quite a few places:

>

>

>

>https://ranger.apache.org/quick_start_guide.html

>


>https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Gui

>de

>

>https://github.com/apache/ranger

>

>

>

>There are other places also.

>

>

>

>Thanks

>

>

>

>Bosco

>

>

>

>

>

>

>

>

>

Ranger mvn build parameters...

[Don Bosco Durai]

Seems after https://issues.apache.org/jira/browse/RANGER-2243 patch, we need to 
pass -Pall to the build. 

 

Without that, I am getting this error:

 

mvn -DskipTests=true clean compile package install assembly:assembly

[ERROR] Could not find goal 'assembly' in plugin 
org.apache.maven.plugins:maven-assembly-plugin:3.0.0 among available goals 
help, single -> [Help 1]

 

Can we change the public documentation with additional details? Seems we need 
to change quite a few places:

 

https://ranger.apache.org/quick_start_guide.html

https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide

https://github.com/apache/ranger

 

There are other places also.

 

Thanks

 

Bosco

 

 

 

 

[jira] [Closed] (RANGER-698) Ranger policy should support variables like $user

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai closed RANGER-698.
--

> Ranger policy should support variables like $user
> -
>
> Key: RANGER-698
> URL: https://issues.apache.org/jira/browse/RANGER-698
> Project: Ranger
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>    Reporter: Don Bosco Durai
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: 0.7.0, 0.6.3
>
> Attachments: RANGER-698.1.patch
>
>
> It would be good to support variables in resources and users.
> E.g.
> HDFS Resource =  /home/$user  
> or
> Table Resource = ${user}_*
> Users allowed = $user
> Where $user will be expanded to the current user. 
> I think, resource substitution will be easy. For permission, we can use key 
> word like we use for all users group="public". We can use key word like 
> "USER" or something like that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-729) Support multiple BaseDNs while syncing users from AD/LDAP

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657925#comment-16657925
 ] 

Don Bosco Durai commented on RANGER-729:


[~spolavarapu]

I think this has been already addressed. If so, we can close this.

Thanks

> Support multiple BaseDNs while syncing users from AD/LDAP
> -
>
> Key: RANGER-729
> URL: https://issues.apache.org/jira/browse/RANGER-729
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 0.5.0
>    Reporter: Don Bosco Durai
>Assignee: Sailaja Polavarapu
>Priority: Major
>
> Since AD search has limitation for not allowing OU/DN in search filter, it is 
> not possible to sync users from parallel OUs.
> Eg: We should support synchronizing users from below search bases
> OU=ServiceUsers,DC=EXAMPLE,DC=COM 
> and 
> OU=CorpUsers,DC=EXAMPLE,DC=COM
> My suggestion for now will be to the take multiple BaseDNs with well known 
> delimiter or space



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (RANGER-720) Ldap discovery tool doesn't seem to be working as expected

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai closed RANGER-720.
--

> Ldap discovery tool doesn't seem to be working as expected
> --
>
> Key: RANGER-720
> URL: https://issues.apache.org/jira/browse/RANGER-720
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 0.5.1
>    Reporter: Don Bosco Durai
>Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 0.5.1, 0.6.0
>
>
> [~spolavarapu]
> I was testing the ldap discovery tool against AD and it seems the results 
> were not as I expected:
> input.properties:
> ranger.usersync.ldap.url=ldap://ad-hello.cloud.hello.com  
>
> ranger.usersync.ldap.binddn=CN=LDAP Access,OU=MyUsers,DC=AD-HELLO,DC=COM
> ranger.usersync.ldap.ldapbindpassword=
> ranger.admin.auth.sampleuser=CN=sample,OU=MyUsers,DC=AD-HELLO,DC=COM
> ranger.admin.auth.samplepassword=
> output:
> SYNC_LDAP_USER_NAME_ATTRIBUTE=sAMAccountName
> SYNC_LDAP_USER_OBJECT_CLASS=person
> SYNC_LDAP_USER_GROUP_NAME_ATTRIBUTE=
> SYNC_LDAP_USER_SEARCH_BASE=OU=workshop_service_users,DC=AD-HDP,DC=COM
> SYNC_LDAP_USER_SEARCH_FILTER=sAMAccountName=*
> ldapConfigCheck.log
> INFO: No. of users from DC=AD-HELLO,DC=COM = 1
> INFO: No. of users from OU=workshop_service_users,DC=AD-HELLO,DC=COM = 12
> INFO: No. of users from OU=MyUsers,DC=AD-HELLO,DC=COM = 1
> INFO: No. of users from OU=Domain Controllers,DC=AD-HELLO,DC=COM = 1
> INFO: No. of users from CN=Users,DC=AD-HELLO,DC=COM = 5
> INFO: No. of users from DC=AD-HELLO,DC=COM = 1
> INFO: No. of users from OU=workshop_service_users,DC=AD-HELLO,DC=COM = 12
> INFO: No. of users from OU=MyUsers,DC=AD-HELLO,DC=COM = 1
> INFO: No. of users from OU=Domain Controllers,DC=AD-HELLO,DC=COM = 1
> INFO: No. of users from CN=Users,DC=AD-HELLO,DC=COM = 5
> ERROR: Connection failed: null
> I was expecting the following:
> SYNC_LDAP_USER_GROUP_NAME_ATTRIBUTE=sAMAccountName
> SYNC_LDAP_USER_SEARCH_BASE=OU=MyUsers,DC=AD-HELLO,DC=COM
> Also, there is an ERROR: Connection failed: null
> Let me know if you need additional information. Thanks



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (RANGER-729) Support multiple BaseDNs while syncing users from AD/LDAP

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai reassigned RANGER-729:
--

Assignee: Sailaja Polavarapu

> Support multiple BaseDNs while syncing users from AD/LDAP
> -
>
> Key: RANGER-729
> URL: https://issues.apache.org/jira/browse/RANGER-729
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 0.5.0
>    Reporter: Don Bosco Durai
>Assignee: Sailaja Polavarapu
>Priority: Major
>
> Since AD search has limitation for not allowing OU/DN in search filter, it is 
> not possible to sync users from parallel OUs.
> Eg: We should support synchronizing users from below search bases
> OU=ServiceUsers,DC=EXAMPLE,DC=COM 
> and 
> OU=CorpUsers,DC=EXAMPLE,DC=COM
> My suggestion for now will be to the take multiple BaseDNs with well known 
> delimiter or space



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (RANGER-813) Script to install Solr for Ranger Audits doesn't work in Suse

[Don Bosco Durai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Don Bosco Durai closed RANGER-813.
--

> Script to install Solr for Ranger Audits doesn't work in Suse
> -
>
> Key: RANGER-813
> URL: https://issues.apache.org/jira/browse/RANGER-813
> Project: Ranger
>  Issue Type: Bug
>    Reporter: Don Bosco Durai
>    Assignee: Don Bosco Durai
>Priority: Major
> Fix For: 1.0.0
>
>
> Seems suse OS doesn't have adduser utility method to add users.
> In the ranger script to install solr, we will need to use "useradd" instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)