Review Request 55959: RANGER-1307:Enable Deny and Exclusions conditions in Ranger Policies for WASB service-def

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55959/
---

Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Repository: ranger


Description
---

RANGER-1307:Enable Deny and Exclusions conditions in Ranger Policies for WASB 
service-def


Diffs
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-wasb.json 
b6f600c 

Diff: https://reviews.apache.org/r/55959/diff/


Testing
---

Tested in local VM


Thanks,

Ramesh Mani

Review Request 55961: RANGER-1223:Ranger doesn't have the correct error message when audit configuration file is not present

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55961/
---

Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Repository: ranger


Description
---

RANGER-1223:Ranger doesn't have the correct error message when audit 
configuration file is not present


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/authorization/hadoop/config/RangerLegacyConfigBuilder.java
 3b0a3fc 

Diff: https://reviews.apache.org/r/55961/diff/


Testing
---

tested in local vm


Thanks,

Ramesh Mani

Re: [VOTE] Release Apache Ranger 0.6.3 - release candidate 1 (dev group vote)

[Ramesh Mani]

+1 (binding) 

Did full maven build of source from
https://dist.apache.org/repos/dist/dev/ranger/0.6.3-rc1/apache-ranger-0.6.3
.tar.gz without issue.

Verified some of the src files.

Thanks,
Ramesh



On 1/26/17, 12:50 PM, "Velmurugan Periasamy"  wrote:

>Rangers:
>
>Apache Ranger 0.6.3 release candidate #1 is now available for a vote
>within dev community. Links to release artifacts are given below. Could
>you please review and vote? Please note that this vote is being redone
>after Ranger has graduated from incubator to a TLP.
>
>The vote will be open for at least 72 hours or until necessary number of
>votes are reached.
>[ ] +1  approve
>[ ] +0  no opinion
>[ ] -1  disapprove (and reason why)
>
>Here is my +1
>
>Thank you,
>Vel
>
>Git tag for the release:
>  https://github.com/apache/ranger/tree/ranger-0.6.3-rc1  (last commit id
>:  bedbc4bda97b54113e166307596d8c62ce5d329f)
>Sources for the release:
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.6.3-rc1/apache-ranger-0.6.
>3.tar.gz
>
>Source release verification:
>PGP Signature:  
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.6.3-rc1/apache-ranger-0.6.
>3.tar.gz.asc 
>MD5/SHA  Hash:
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.6.3-rc1/apache-ranger-0.6.
>3.tar.gz.mds 
>
>Keys to verify the signature of the release artifact are available at:
>  https://people.apache.org/keys/group/ranger.asc
>
>

Re: Review Request 56146: RANGER-1342: Hive test connection is not working when using local/ldap user credentials in repo config

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56146/#review163769
---


Ship it!




Ship It!

- Ramesh Mani


On Feb. 1, 2017, 1:24 a.m., Sailaja Polavarapu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56146/
> ---
> 
> (Updated Feb. 1, 2017, 1:24 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Abhay 
> Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-1342
> https://issues.apache.org/jira/browse/RANGER-1342
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> When reading the config from the service request, the password is not 
> encrypted. But during the client login, code expects the password in 
> encrypted format. This issue happens with other plugins like hdfs, yarn, and 
> hbase as well.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/client/HadoopConfigHolder.java
>  902a8b9 
> 
> Diff: https://reviews.apache.org/r/56146/diff/
> 
> 
> Testing
> ---
> 
> 1. Performed functional tests with test connection as well as resource lookup 
> as the code flow is common for both the cases.
> 2. Also, verified cases for other plugins like hbase and hdfs
> 3. Verified both the cases with and without saving new credentials in the 
> repo config.
> 4. Verfied with both local and AD users.
> 
> 
> Thanks,
> 
> Sailaja Polavarapu
> 
>

Review Request 56192: RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is used to set the UGI instead of Principal/Keytab

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56192/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Bugs: RANGER-1338
https://issues.apache.org/jira/browse/RANGER-1338


Repository: ranger


Description
---

RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is 
used to set the UGI instead of Principal/Keytab


Diffs
-

  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
d440b85 
  
knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
 2ec5300 
  
storm-agent/src/main/java/org/apache/ranger/authorization/storm/authorizer/RangerStormAuthorizer.java
 c66b665 

Diff: https://reviews.apache.org/r/56192/diff/


Testing
---


Thanks,

Ramesh Mani

Re: Review Request 56192: RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is used to set the UGI instead of Principal/Keytab

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56192/
---

(Updated Feb. 1, 2017, 10:35 p.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Changes
---

Minor change in the code


Bugs: RANGER-1338
https://issues.apache.org/jira/browse/RANGER-1338


Repository: ranger


Description
---

RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is 
used to set the UGI instead of Principal/Keytab


Diffs (updated)
-

  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
d440b85 
  
knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
 2ec5300 
  
storm-agent/src/main/java/org/apache/ranger/authorization/storm/authorizer/RangerStormAuthorizer.java
 c66b665 

Diff: https://reviews.apache.org/r/56192/diff/


Testing
---


Thanks,

Ramesh Mani

Re: Review Request 56192: RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is used to set the UGI instead of Principal/Keytab

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56192/
---

(Updated Feb. 2, 2017, 1:27 a.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Changes
---

Fixed review comments


Bugs: RANGER-1338
https://issues.apache.org/jira/browse/RANGER-1338


Repository: ranger


Description
---

RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is 
used to set the UGI instead of Principal/Keytab


Diffs (updated)
-

  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
d440b85 
  
knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
 2ec5300 
  
storm-agent/src/main/java/org/apache/ranger/authorization/storm/authorizer/RangerStormAuthorizer.java
 c66b665 

Diff: https://reviews.apache.org/r/56192/diff/


Testing (updated)
---

Testing done in Local VM


Thanks,

Ramesh Mani

Re: Review Request 56192: RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is used to set the UGI instead of Principal/Keytab

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56192/
---

(Updated Feb. 4, 2017, 5:37 a.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Changes
---

Add Warning in case of simple cluster instead of showing exception if 
JAASConfig is not found


Bugs: RANGER-1338
https://issues.apache.org/jira/browse/RANGER-1338


Repository: ranger


Description
---

RANGER-1338:Ranger Plugin failed to download policy when JaasConfig alone is 
used to set the UGI instead of Principal/Keytab


Diffs (updated)
-

  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
bb85e5e 

Diff: https://reviews.apache.org/r/56192/diff/


Testing
---

Testing done in Local VM


Thanks,

Ramesh Mani

Review Request 56335: Ranger Audit framework enhancement to provide an option to allow audit records to be spooled to local disk first before sending it to destinations

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56335/
---

Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, 
and Velmurugan Periasamy.


Repository: ranger


Description
---

Ranger Audit framework enhancement to provide an option to allow audit records 
to be spooled to local disk first before sending it to destinations


Diffs
-

  
agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
 7c37cfa 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
 PRE-CREATION 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditProviderFactory.java
 e3c3508 
  
agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/56335/diff/


Testing
---

Test all the plugins in Local VM
To enable the file cache provider for each of the components please do the 
following

For HDFS Plugin
===
mkdir -p  /var/log/hadoop/hdfs/audit/spool
cd /var/log/hadoop/hdfs/audit/
chown hdfs:hadoop spool
Add the following properties to the "custom ranger-hive-audit” in the 
Ambari for hdfs. 
xasecure.audit.provider.filecache.is.enabled=true
xasecure.audit.provider.filecache.filespool.file.rollover.sec=300

xasecure.audit.provider.filecache.filespool.dir=/var/log/hadoop/hdfs/audit/spool

   NOTE:
xasecure.audit.provider.filecache.is.enabled = true 
   This property will enable file cache provider which will store the 
audit locally first before sending it to destinations to avoid lose of data 
xasecure.audit.provider.filecache.filespool.file.rollover.sec=300
This property will close each of local file every 300 sec ( 5 min ) 
and send it destinations. For testing we maded to 30 sec.

xasecure.audit.provider.filecache.filespool.dir=/var/log/hadoop/hdfs/audit/spool
  This property is the directory where the local audit cache is 
present.

For Hive Plugin
=

   mkdir -p /var/log/hive/audit/spool
cd /var/log/hive/audit/
chown hdfs:hadoop spool
Add the following properties to the "custom ranger-hive-audit” in the Ambari 
for hdfs. 
xasecure.audit.provider.filecache.is.enabled=true
xasecure.audit.provider.filecache.filespool.file.rollover.sec=300
xasecure.audit.provider.filecache.filespool.dir=/var/log/hive/audit/spool

Please do the same steps mentioned  for all the components which  need this 
audit file cache provider.


---
Issues:
- Audit to HDFS destination gets 0 bytes file or missing records in the 
file from HDFS plugin when HDFS get restarted and 
   audit from hdfs plugin is logged into destination.
  
- Audit to HDFS destination gets partial records from 
HIVE/HBASE/KNOX/STORM plugin when HDFS is restarted and there are active 
spooling into hdfs is happening.

Scenarios to test

1) Audit to HDFS / Solr destination with FileCache enabled- 
HDFS/HIVESERVER2/HBASE/KNOX/STORM/KAFKA.
- Mentioned issue should not happen.
- Audit will be getting pushed every 5 minutes ( we are setting 
it to 300 sec in the parameter)

2) Audit to HDFS / Solr destination with FileCache enabled  with one of the 
destination is down and brought back up later.
- Audit from the local cache should be present in destination 
when the destination is up 
- In case of HDFS as destination audit might show up during 
next rollover of hdfs file or  if the corresponding component
   is restarted ( say if it is hiveserver2 plugin, when 
Hiveserver2 is restarted audit into HDFS appears as this will close the
existing opened hdfsfile)
 - Mentioned issue should not be present
 - 
 - 
3) Same has to be done for each for the plugins ( HBASE, STORM, KAFKA, KMS)


Thanks,

Ramesh Mani

Review Request 56405: RANGER-1361:RangerHDFSPlugin audits for Ancestor, Sub level and parent access doesn't have the correct accessType

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56405/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Repository: ranger


Description
---

RANGER-1361:RangerHDFSPlugin audits for Ancestor, Sub level and parent access 
doesn't have the correct accessType


Diffs
-

  
hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
 d92bf12 

Diff: https://reviews.apache.org/r/56405/diff/


Testing
---

Tested HDFS Plugin in local VM


Thanks,

Ramesh Mani

Review Request 56406: RANGER-1355:Ranger HiveAuthorizer should check for ALTER permission for MSCK command while authorizing

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56406/
---

Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Repository: ranger


Description
---

RANGER-1355:Ranger HiveAuthorizer should check for ALTER permission for MSCK 
command while authorizing


Diffs
-

  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 c107b4b 

Diff: https://reviews.apache.org/r/56406/diff/


Testing
---

Tested in local vm


Thanks,

Ramesh Mani

Review Request 56456: RANGER-1320:Ranger Hive Plugin Exception message correction

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56456/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Selvamohan 
Neethiraj, and Velmurugan Periasamy.


Repository: ranger


Description
---

RANGER-1320:Ranger Hive Plugin Exception message correction


Diffs
-

  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 c107b4b 

Diff: https://reviews.apache.org/r/56456/diff/


Testing
---

Currently when  select query fail in authorizaton, Exception message shows all 
the columns in it and it is concern that we are revealing all the columns. With 
this we don't show the columns.
Tested in local VM


Thanks,

Ramesh Mani

Re: Review Request 56461: Modify Ranger Hbase Plugin ColumnIterator to use Cell instead of KeyValue (to avoid ClassCastException in certain cases)

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56461/#review164953
---


Ship it!




Ship It!

- Ramesh Mani


On Feb. 8, 2017, 8:23 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56461/
> ---
> 
> (Updated Feb. 8, 2017, 8:23 p.m.)
> 
> 
> Review request for ranger, Alok Lal, Madhan Neethiraj, Ramesh Mani, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1365
> https://issues.apache.org/jira/browse/RANGER-1365
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Modified ColumnIterator to use interface 'Cell' instead of 'KeyValue', a 
> specific subclass of 'Cell'
> 
> 
> Diffs
> -
> 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/ColumnIterator.java
>  b4e 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>  5dd727b 
>   
> hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/ColumnIteratorTest.java
>  4b40038 
> 
> Diff: https://reviews.apache.org/r/56461/diff/
> 
> 
> Testing
> ---
> 
> Passed all unit tests including modified test for ColumnIterator.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Last date for submitting abstracts - DW summit

[Ramesh Mani]

Balaji,

I have submitted now a Abstract on how to "Extending Ranger Authorization
Model to other Applications²

Thanks,
Ramesh

On 2/10/17, 10:03 AM, "Balaji Ganesan"  wrote:

>Rangers,
>
>Today is the deadline for submitting abstracts to Data Works/Hadoop Summit
>sessions in San Jose.
>
>https://dataworkssummit.com/san-jose-2017/abstracts/submit-abstract/
>
>It is a good opportunity to talk about your work and experience using
>Ranger,  and share it with the community.
>
>Thanks,
>Balaji

Re: Review Request 56335: RANGER-1310: Ranger Audit framework enhancement to provide an option to allow audit records to be spooled to local disk first before sending it to destinations

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56335/
---

(Updated Feb. 10, 2017, 10:50 p.m.)


Review request for ranger, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, 
and Velmurugan Periasamy.


Changes
---

Bug number and Branch added


Summary (updated)
-

 RANGER-1310: Ranger Audit framework enhancement to provide an option to allow 
audit records to be spooled to local disk first before sending it to 
destinations


Bugs: RANGER-1310
https://issues.apache.org/jira/browse/RANGER-1310


Repository: ranger


Description (updated)
---

RANGER-1310: Ranger Audit framework enhancement to provide an option to allow 
audit records to be spooled to local disk first before sending it to 
destinations


Diffs
-

  
agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
 7c37cfa 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
 PRE-CREATION 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditProviderFactory.java
 e3c3508 
  
agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/56335/diff/


Testing
---

Test all the plugins in Local VM
To enable the file cache provider for each of the components please do the 
following

For HDFS Plugin
===
mkdir -p  /var/log/hadoop/hdfs/audit/spool
cd /var/log/hadoop/hdfs/audit/
chown hdfs:hadoop spool
Add the following properties to the "custom ranger-hive-audit” in the 
Ambari for hdfs. 
xasecure.audit.provider.filecache.is.enabled=true
xasecure.audit.provider.filecache.filespool.file.rollover.sec=300

xasecure.audit.provider.filecache.filespool.dir=/var/log/hadoop/hdfs/audit/spool

   NOTE:
xasecure.audit.provider.filecache.is.enabled = true 
   This property will enable file cache provider which will store the 
audit locally first before sending it to destinations to avoid lose of data 
xasecure.audit.provider.filecache.filespool.file.rollover.sec=300
This property will close each of local file every 300 sec ( 5 min ) 
and send it destinations. For testing we maded to 30 sec.

xasecure.audit.provider.filecache.filespool.dir=/var/log/hadoop/hdfs/audit/spool
  This property is the directory where the local audit cache is 
present.

For Hive Plugin
=

   mkdir -p /var/log/hive/audit/spool
cd /var/log/hive/audit/
chown hdfs:hadoop spool
Add the following properties to the "custom ranger-hive-audit” in the Ambari 
for hdfs. 
xasecure.audit.provider.filecache.is.enabled=true
xasecure.audit.provider.filecache.filespool.file.rollover.sec=300
xasecure.audit.provider.filecache.filespool.dir=/var/log/hive/audit/spool

Please do the same steps mentioned  for all the components which  need this 
audit file cache provider.


---
Issues:
- Audit to HDFS destination gets 0 bytes file or missing records in the 
file from HDFS plugin when HDFS get restarted and 
   audit from hdfs plugin is logged into destination.
  
- Audit to HDFS destination gets partial records from 
HIVE/HBASE/KNOX/STORM plugin when HDFS is restarted and there are active 
spooling into hdfs is happening.

Scenarios to test

1) Audit to HDFS / Solr destination with FileCache enabled- 
HDFS/HIVESERVER2/HBASE/KNOX/STORM/KAFKA.
- Mentioned issue should not happen.
- Audit will be getting pushed every 5 minutes ( we are setting 
it to 300 sec in the parameter)

2) Audit to HDFS / Solr destination with FileCache enabled  with one of the 
destination is down and brought back up later.
- Audit from the local cache should be present in destination 
when the destination is up 
- In case of HDFS as destination audit might show up during 
next rollover of hdfs file or  if the corresponding component
   is restarted ( say if it is hiveserver2 plugin, when 
Hiveserver2 is restarted audit into HDFS appears as this will close the
existing opened hdfsfile)
 - Mentioned issue should not be present
 - 
 - 
3) Same has to be done for each for the plugins ( HBASE, STORM, KAFKA, KMS)


Thanks,

Ramesh Mani

Review Request 56571: RANGER-1321:Provide a mechanism to create service-specific default policies

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56571/
---

Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Bugs: RANGER-1321
https://issues.apache.org/jira/browse/RANGER-1321


Repository: ranger


Description
---

RANGER-1321:Provide a mechanism to create service-specific default policies


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBaseService.java
 73027a0 
  security-admin/pom.xml 0fcfc5a 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
cb67b6a 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
2b773da 

Diff: https://reviews.apache.org/r/56571/diff/


Testing
---

Test in Local VM


Thanks,

Ramesh Mani

Review Request 56612: RANGER-1382:Good coding practice in Ranger recommended by static code analysis

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56612/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Bugs: Ranger-1382
https://issues.apache.org/jira/browse/Ranger-1382


Repository: ranger


Description
---

RANGER-1382:Good coding practice in Ranger recommended by static code analysis


Diffs
-

  
agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
 94db401 

Diff: https://reviews.apache.org/r/56612/diff/


Testing
---

Tested in Local Vm


Thanks,

Ramesh Mani

Review Request 56614: RANGER-1381:Add hadoop-common.jar as dependency to ranger-hive-utils package to avoid build failure

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56614/
---

Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Bugs: RANGER-1381
https://issues.apache.org/jira/browse/RANGER-1381


Repository: ranger


Description
---

RANGER-1381:Add hadoop-common.jar as dependency to ranger-hive-utils package to 
avoid build failure


Diffs
-

  ranger-hive-utils/pom.xml 9d32f66 

Diff: https://reviews.apache.org/r/56614/diff/


Testing
---

Testing done in Local build


Thanks,

Ramesh Mani

Re: Review Request 56658: RANGER-1384 - Replace old asm version

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56658/#review165626
---


Ship it!




Ship It!

- Ramesh Mani


On Feb. 14, 2017, 11:55 a.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56658/
> ---
> 
> (Updated Feb. 14, 2017, 11:55 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1384
> https://issues.apache.org/jira/browse/RANGER-1384
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1376 upgrades some Jersey dependencies, and causes a problem with the 
> security admin as the "asm/asm" dependency is no longer getting pulled in 
> from Jersey. We should remove the older "org.ow2.util.asm" in the 
> security-admin, and just pull "asm/asm-all" in instead, for consistency with 
> the kms module.
> 
> 
> Diffs
> -
> 
>   pom.xml 4f83391 
>   security-admin/pom.xml bdc5982 
> 
> Diff: https://reviews.apache.org/r/56658/diff/
> 
> 
> Testing
> ---
> 
> Tested that the Ranger Admin starts OK with both this patch and the Jersey 
> patch for RANGER-1376.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 56571: RANGER-1321:Provide a mechanism to create service-specific default policies

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56571/
---

(Updated Feb. 18, 2017, 2:06 a.m.)


Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Changes
---

Review comment fixed


Bugs: RANGER-1321
https://issues.apache.org/jira/browse/RANGER-1321


Repository: ranger


Description
---

RANGER-1321:Provide a mechanism to create service-specific default policies


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBaseService.java
 73027a0 
  
agents-common/src/main/java/org/apache/ranger/services/tag/RangerServiceTag.java
 3940154 
  plugin-kms/src/main/java/org/apache/ranger/services/kms/RangerServiceKMS.java 
8b41ae3 
  security-admin/pom.xml 0fcfc5a 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
cb67b6a 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
2b773da 

Diff: https://reviews.apache.org/r/56571/diff/


Testing
---

Test in Local VM


Thanks,

Ramesh Mani

Re: [VOTE] Apache Ranger Release 0.7.0 - rc1

[Ramesh Mani]

+1 (binding)

- Did successful build of apache-ranger-0.7.0.tar.gz  with rat plugin.
- Verified PGP signature, MD5/SHA hash.

Thanks,
Ramesh


On 2/20/17, 10:21 AM, "Selvamohan Neethiraj"  wrote:

>Rangers:
>
> 
>
>Thank you for your contribution to this Apache community to be able to
>release this major release ­ Ranger 0.7.0.
>
>This release includes numerous improvements/bug-fixes as well as some new
>features over the previous release of Ranger (see Release Notes for
>details).
>
> 
>
>Apache ranger-0.7.0 release candidate # 1 is now available with the
>following artifacts up for vote.
>
>I kindly request all of the Rangers (dev & PMC members) to review and
>vote on this release.
>
> 
>
>Git tag for the release:
>
>https://github.com/apache/ranger/tree/ranger-0.7.0-rc1 (last commit id:
>f39e4880082f5579db49b29be735977a341c041b)
>
> 
>
>Sources for the release:
>
>https://dist.apache.org/repos/dist/dev/ranger/0.7.0-rc1/apache-ranger-0.7.
>0.tar.gz
>
> 
>
>Source release verification:
>
>PGP Signature:  
>
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.7.0-rc1/apache-ranger-0.7.
>0.tar.gz.asc
>
>MD5/SHA Hash:
>
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.7.0-rc1/apache-ranger-0.7.
>0.tar.gz.mds
>
> 
>
>Keys to verify the signature of the release artifact are available at:
>
>  https://people.apache.org/keys/group/ranger.asc
>
> 
>
>Release Notes:
>
>   
>https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.7.0+-+R
>elease+Notes
>
> 
>
>Build verification steps can be found at:
>
>   http://ranger.apache.org/quick_start_guide.html
>
> 
>
>The vote will be open for at least 72 hours or until necessary number of
>votes are reached.
>
>[ ] +1  approve
>
>[ ] +0  no opinion
>
>[ ] -1  disapprove (and reason why)
>
> 
>
>Here is my +1 (binding).
>
> 
>
>Thank you,
>
>Selva-
>

Review Request 57002: RANGER-1406: Audit spoolfile not getting created when ranger service user didn't have permission to log into Solr

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57002/
---

Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Bugs: RANGER-1406
https://issues.apache.org/jira/browse/RANGER-1406


Repository: ranger


Description
---

RANGER-1406: Audit spoolfile not getting created when ranger service user 
didn't have permission to log into Solr


Diffs
-

  
agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java
 405cfee 

Diff: https://reviews.apache.org/r/57002/diff/


Testing
---

Testing done in local


Thanks,

Ramesh Mani

Re: Review Request 57002: RANGER-1406: Audit spoolfile not getting created when ranger service user didn't have permission to log into Solr

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57002/
---

(Updated Feb. 24, 2017, 7:03 a.m.)


Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Changes
---

Changes done after review


Bugs: RANGER-1406
https://issues.apache.org/jira/browse/RANGER-1406


Repository: ranger


Description
---

RANGER-1406: Audit spoolfile not getting created when ranger service user 
didn't have permission to log into Solr


Diffs (updated)
-

  
agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java
 405cfee 

Diff: https://reviews.apache.org/r/57002/diff/


Testing (updated)
---

Testing done in local vm


Thanks,

Ramesh Mani

Re: Review Request 56571: RANGER-1321:Provide a mechanism to create service-specific default policies

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56571/
---

(Updated Feb. 24, 2017, 8:19 a.m.)


Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Changes
---

Fixed review comments


Bugs: RANGER-1321
https://issues.apache.org/jira/browse/RANGER-1321


Repository: ranger


Description
---

RANGER-1321:Provide a mechanism to create service-specific default policies


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBaseService.java
 73027a0 
  
agents-common/src/main/java/org/apache/ranger/services/tag/RangerServiceTag.java
 3940154 
  plugin-kms/src/main/java/org/apache/ranger/services/kms/RangerServiceKMS.java 
8b41ae3 
  security-admin/pom.xml 0fcfc5a 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
cb67b6a 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
2b773da 

Diff: https://reviews.apache.org/r/56571/diff/


Testing
---

Test in Local VM


Thanks,

Ramesh Mani

Re: Review Request 57099: Good coding practice in Ranger recommended by static code analysis

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57099/#review166923
---


Fix it, then Ship it!




Fix it and ship it


agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java
 (line 137)
<https://reviews.apache.org/r/57099/#comment239009>

class level object need not be initialized


- Ramesh Mani


On Feb. 27, 2017, 7:03 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57099/
> ---
> 
> (Updated Feb. 27, 2017, 7:03 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-1413
> https://issues.apache.org/jira/browse/RANGER-1413
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Fix potention run-time issues uncovered by static code analysis.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java
>  f4d76ad 
> 
> Diff: https://reviews.apache.org/r/57099/diff/
> 
> 
> Testing
> ---
> 
> Ran unit tests successfully.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 57099: Good coding practice in Ranger recommended by static code analysis

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57099/#review166924
---


Ship it!




Ship It!

- Ramesh Mani


On Feb. 27, 2017, 7:43 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57099/
> ---
> 
> (Updated Feb. 27, 2017, 7:43 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-1413
> https://issues.apache.org/jira/browse/RANGER-1413
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Fix potention run-time issues uncovered by static code analysis.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/resourcematcher/TestDefaultPolicyResourceMatcherForPolicy.java
>  f4d76ad 
> 
> Diff: https://reviews.apache.org/r/57099/diff/
> 
> 
> Testing
> ---
> 
> Ran unit tests successfully.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 57123: RANGER-1407 : Service update transaction log is not generated in some cases

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57123/#review167213
---




security-admin/src/main/java/org/apache/ranger/service/RangerServiceService.java
Lines 265 (patched)
<https://reviews.apache.org/r/57123/#comment239377>

Please consider doing this check as "tagservice".equalsIgnoreCase(fieldName)



security-admin/src/main/java/org/apache/ranger/service/RangerServiceService.java
Lines 266 (patched)
<https://reviews.apache.org/r/57123/#comment239378>

!"null".equalsIgnoreCase(oldValue) is the check done because oldValue will 
have a string "null". if this check is only for null value please remove it.


- Ramesh Mani


On March 1, 2017, 6:24 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57123/
> ---
> 
> (Updated March 1, 2017, 6:24 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Selvamohan Neethiraj, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1407
> https://issues.apache.org/jira/browse/RANGER-1407
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** If a Ranger service is updated to change its 
> associated tag service name, then the service-update transaction log is not 
> generated.
> 
> **Proposed Solution :** 'tagService' attribute of RangerService class need to 
> be enabled for logging the changes in value of this attribute in 
> RangerServiceService class.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> a75c19a 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerServiceService.java
>  004524b 
> 
> 
> Diff: https://reviews.apache.org/r/57123/diff/2/
> 
> 
> Testing
> ---
> 
> **Steps performed with patch : **
> 1. Installed and Started Ranger admin.
> 2. Created hdfs service 'hdfsdev'
> 3. Created Tagbased service 'tagdev'
> 4. Clicked on edit button of 'hdfsdev' service and updated tag service name 
> to 'tagdev'
> 5. tag service name 'tagdev' got updated in 'hdfsdev'
> 6. visited Audit menu->Admin tab to check whether log for the tag service 
> name update event is appearing or not.
> 
> **Expected Behaviour : **
> Tag service name update log should appear under the Admin tab.
> 
> **Actual Behaviour : **
> 1. Tag service name update log was appearring under the Admin tab and after 
> clicking on the log was able to see log details like given below : 
> 
> Fields : Tag Service Name
> Old Value : --
> New Value :tagdev
> 
> 2. Created tag service 'tagdev1' and after changing tag service name of 
> 'hdfsdev' to 'tagdev1 was able to see log details like given below : 
> 
> Fields : Tag Service Name
> Old Value : tagdev
> New Value :tagdev1
> 
> 3. Also was able to see logs of removal of tag service name 'tagdev1' from 
> service 'hdfsdev' like given below :
> Fields : Tag Service Name
> Old Value : tagdev1
> New Value : --
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 57123: RANGER-1407 : Service update transaction log is not generated in some cases

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57123/#review167343
---


Ship it!




Ship It!

- Ramesh Mani


On March 1, 2017, 8:52 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57123/
> ---
> 
> (Updated March 1, 2017, 8:52 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Selvamohan Neethiraj, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1407
> https://issues.apache.org/jira/browse/RANGER-1407
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** If a Ranger service is updated to change its 
> associated tag service name, then the service-update transaction log is not 
> generated.
> 
> **Proposed Solution :** 'tagService' attribute of RangerService class need to 
> be enabled for logging the changes in value of this attribute in 
> RangerServiceService class.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> a75c19a 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerServiceService.java
>  004524b 
> 
> 
> Diff: https://reviews.apache.org/r/57123/diff/3/
> 
> 
> Testing
> ---
> 
> **Steps performed with patch : **
> 1. Installed and Started Ranger admin.
> 2. Created hdfs service 'hdfsdev'
> 3. Created Tagbased service 'tagdev'
> 4. Clicked on edit button of 'hdfsdev' service and updated tag service name 
> to 'tagdev'
> 5. tag service name 'tagdev' got updated in 'hdfsdev'
> 6. visited Audit menu->Admin tab to check whether log for the tag service 
> name update event is appearing or not.
> 
> **Expected Behaviour : **
> Tag service name update log should appear under the Admin tab.
> 
> **Actual Behaviour : **
> 1. Tag service name update log was appearring under the Admin tab and after 
> clicking on the log was able to see log details like given below : 
> 
> Fields : Tag Service Name
> Old Value : --
> New Value :tagdev
> 
> 2. Created tag service 'tagdev1' and after changing tag service name of 
> 'hdfsdev' to 'tagdev1 was able to see log details like given below : 
> 
> Fields : Tag Service Name
> Old Value : tagdev
> New Value :tagdev1
> 
> 3. Also was able to see logs of removal of tag service name 'tagdev1' from 
> service 'hdfsdev' like given below :
> Fields : Tag Service Name
> Old Value : tagdev1
> New Value : --
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Review Request 57301: RANGER-1422:Ranger Knox Plugin audit doesn't have the access type populated

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57301/
---

Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
Polavarapu, and Velmurugan Periasamy.


Bugs: RANGER-1422
https://issues.apache.org/jira/browse/RANGER-1422


Repository: ranger


Description
---

RANGER-1422:Ranger Knox Plugin audit doesn't have the access type populated


Diffs
-

  
knox-agent/src/main/java/org/apache/ranger/authorization/knox/KnoxRangerPlugin.java
 871e53a 


Diff: https://reviews.apache.org/r/57301/diff/1/


Testing
---

do knox operation like liststatus via webhdfs and generate some audit. 
Resulting audit that you see in ranger admin will have the access type as allow


Thanks,

Ramesh Mani

Re: Review Request 57239: RANGER-1416 - SunX509 is the hardcoded Algorithm for SSL

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57239/#review167904
---


Ship it!




verified the patch.

- Ramesh Mani


On March 2, 2017, 10:18 a.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57239/
> ---
> 
> (Updated March 2, 2017, 10:18 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1416
> https://issues.apache.org/jira/browse/RANGER-1416
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> In IBM JDK environment, the SSL Algorithm is IbmX509. The hardcoded SunX509 
> fails in the IBM JDK environment. This is similar to the issue for 
> THRIFT-1332: TSSLTransportParameters class uses hard coded value 
> keyManagerType: SunX509
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
>  ba3a82e 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java
>  e0b0fd0 
> 
> 
> Diff: https://reviews.apache.org/r/57239/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Admin Server.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Auditing using Solr

[Ramesh Mani]

Nigel,

There is one option which introduced by
https://issues.apache.org/jira/browse/RANGER-1310.

This will enable you to always spool to local disk first before the audit
is pushed to the destinations. This guarantees that the audit data is not
lost if the memory queue is destroyed by the restart of of the any
components.

Thanks,
Ramesh

On 3/6/17, 9:31 AM, "Don Bosco Durai"  wrote:

>All your assumptions are correct, except.
>
>>* unless the queue size is exceeded in which case events are lost
>The audit framework will automatically start spooling to file if:
>- Queue size is exceeded
>- Destination is down for extended period (I think 10 minutes, but need
>to verify)
>
>Bosco
>
>On 3/6/17, 9:28 AM, "Nigel Jones"  wrote:
>
>I'm planning to use Solr for audit (easy searching, aggregation) and
>trying to understand failure modes
>
>If solr is not ready when the plugin starts up I assume we'll try to
>connect (1s?) then wait for a period (30s) then retry
>
>However this is on an async thread, and meanwhile audit events are
>queues in memory locally ... so
>
>* If solr starts after the plugin, initial events are delayed but
>then 
>will log
>* if solr breaks, events will temporarily pause, but then resume once
>solr back up
>* unless the queue size is exceeded in which case events are lost
>
>is my understanding correct?
>
>Thanks :-)
>Nigel.
>
>
>
>
>

Re: Review Request 57373: In certain scenario user data contains junk email-id

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57373/#review168249
---




security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
Line 2007 (original)
<https://reviews.apache.org/r/57373/#comment240464>

Can this be 
vXPortalUser.setEmailAddress(vXUser.getEmailAddress());


- Ramesh Mani


On March 8, 2017, 4:22 a.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57373/
> ---
> 
> (Updated March 8, 2017, 4:22 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1428
> https://issues.apache.org/jira/browse/RANGER-1428
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Junk email id is displayed in some scenario
> 
> **Steps to reproduce**
> When added non-existing user while creating a service, it gets created in 
> user listing without email id but when same user is deleted it shows some 
> random data in email field in “User Profile deleted” entry
> 
> Steps to follow:
> 1. Create a service with non-existing user(user will get created in user 
> listing)
> 2. Now delete that user
> 3. Go to Audit-Admin tab and check log for "User Profile deleted", in 
> email-id field it will show some random data
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 62cffa5 
> 
> 
> Diff: https://reviews.apache.org/r/57373/diff/2/
> 
> 
> Testing
> ---
> 
> Tested on simple as well as secure cluster
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

Re: Review Request 57303: Refactoring to move default policy creation to RangerService classes

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57303/#review168806
---


Ship it!




Ship It!

- Ramesh Mani


On March 11, 2017, 8:44 a.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57303/
> ---
> 
> (Updated March 11, 2017, 8:44 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-1321
> https://issues.apache.org/jira/browse/RANGER-1321
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Default policy creation is best handled by the Service-specific class. The 
> relevant code is moved out ServiceDBStore and into the RangerServiceBase 
> class. It is expected that subclasses of RangerServiceBase representing each 
> Service, will specialize the default policy creation behavior if necessary by 
> overriding createDefaultPolicies() API.
> 
> RangerServiceKafka needs special handling if the cluster is Kerberized. 
> Currently, there is only a placeholder for it.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBaseService.java
>  debaa83 
>   
> agents-common/src/main/java/org/apache/ranger/services/tag/RangerServiceTag.java
>  bcf9064 
>   
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>  324551d 
>   
> hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/RangerServiceHdfs.java
>  bc12da9 
>   
> plugin-atlas/src/main/java/org/apache/ranger/services/atlas/RangerServiceAtlas.java
>  d2b60bd 
>   
> plugin-kafka/src/main/java/org/apache/ranger/services/kafka/RangerServiceKafka.java
>  86e97bc 
>   
> plugin-kms/src/main/java/org/apache/ranger/services/kms/RangerServiceKMS.java 
> 7657099 
>   
> plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  470c711 
>   
> plugin-yarn/src/main/java/org/apache/ranger/services/yarn/RangerServiceYarn.java
>  69f2bc3 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 053df24 
>   security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
> 531674a 
> 
> 
> Diff: https://reviews.apache.org/r/57303/diff/4/
> 
> 
> Testing
> ---
> 
> Very basic testing done only in a simple cluster.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 57641: RANGER-1458 - Starting Yarn failed after installing Ranger Yarn Plugin

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57641/#review169031
---


Ship it!




Ship It!

- Ramesh Mani


On March 15, 2017, 12:39 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57641/
> ---
> 
> (Updated March 15, 2017, 12:39 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1458
> https://issues.apache.org/jira/browse/RANGER-1458
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Similar to RANGER-1412, starting Yarn after installing the Ranger Yarn plugin 
> fails.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh 9270c5f 
> 
> 
> Diff: https://reviews.apache.org/r/57641/diff/1/
> 
> 
> Testing
> ---
> 
> Tested it with a Hadoop distribution.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 57697: Good coding practice recommended by static code analysis

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57697/#review169198
---


Ship it!




Ship It!

- Ramesh Mani


On March 16, 2017, 6:03 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57697/
> ---
> 
> (Updated March 16, 2017, 6:03 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1463
> https://issues.apache.org/jira/browse/RANGER-1463
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Fix for potential NPE and performance issue
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBaseService.java
>  9955051 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 26080b5 
> 
> 
> Diff: https://reviews.apache.org/r/57697/diff/2/
> 
> 
> Testing
> ---
> 
> Created Ranger service in a local VM and ensured that the default policies 
> are created as expected.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Question about group policies

[Ramesh Mani]

Adding to Abhay comment,

In most of the Ranger Plugin from the components side we use
org.apache.hadoop.security.UserGroupInformation API
https://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/security/UserGr
oupInformation.html which will wrap around JAAS and provides the mechanism
to determine the User and Groups. Please check if this can be used.

Thanks,
Ramesh

On 3/24/17, 12:03 PM, "Abhay Kulkarni"  wrote:

>Hi Alex,
>
>This is exactly right. Users, groups and their associations in Ranger
>(specifically Ranger Admin) are props for being able to define policies.
>They are not the Œsource of truth¹. It is expected that the correct user
><‹-> group associations will be available in the component (service) from
>appropriate authentication system, and provided to Ranger Plugin as part
>of authorization request.
>
>Thanks!
>-Abhay
>
>On 3/24/17, 11:51 AM, "Alexander Denissov"  wrote:
>
>>Hi Ranger experts,
>>
>>We are developing a custom Ranger Plugin for Apache HAWQ(incubating) and
>>noticed that group policies are not behaving as we expected.
>>
>>In Ranger, we define a user U (actually synched from OS). We then
>>manually
>>define group G and enroll user U into it. We then define a policy and
>>grant
>>a privilege to the group G in this policy.
>>
>>On the client side, we do not know that user U belongs to group G, as
>>this
>>information is only defined in Ranger. When we request policy evaluation,
>>we send an empty set for the userGroups API parameter, assuming Ranger
>>will
>>use its internal mapping. But the access is denied by Ranger.
>>
>>So, it seems Ranger will not use the information from its internal user
>><--> group mapping when evaluating policies and would rely on client
>>providing the set of groups for the user explicitly ?
>>
>>This also means user <--> group mapping in Ranger is NOT the source of
>>truth, but rather a mirror of some other authentication system (OS, LDAP,
>>etc) and a service will need to fetch this information upon user
>>authentication and provide to Ranger ?
>>
>>I will appreciate clarification on these points.
>>--
>>Thanks,
>>Alex.
>
>

Re: Review Request 57988: RANGER-1479 : Fix bug in ranger-*-(security/audit).xml loading, which prevented loading from JAR's

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57988/#review170435
---



With this patch there are some pmd violations. Please verify and re-submit the 
patch.
[INFO] PMD Failure: 
org/apache/ranger/authorization/hadoop/config/RangerConfiguration.java:23 
Rule:UnusedImports Priority:4 Avoid unused imports such as 'java.io.File'.
[INFO] PMD Failure: 
org/apache/ranger/authorization/hadoop/config/RangerConfiguration.java:24 
Rule:UnusedImports Priority:4 Avoid unused imports such as 
'java.net.MalformedURLException'.

- Ramesh Mani


On March 29, 2017, 3:34 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57988/
> ---
> 
> (Updated March 29, 2017, 3:34 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1479
> https://issues.apache.org/jira/browse/RANGER-1479
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerConfiguration first checks, if the necessary xml files are on the 
> classpath, and after it tries to convert it to a file path, check that file 
> is on the filesystem, and convert that path to back to URL. If the 
> configurations are inside a jar, then this will fail very badly (the resource 
> url will be something file://config.jar!/ranger-services.xml )
> 
> If the conversion is removed, it would work pretty nicely
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/hadoop/config/RangerConfiguration.java
>  d81f6b9 
> 
> 
> Diff: https://reviews.apache.org/r/57988/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Re: Review Request 58067: Ranger hive service definition to use hive metastore directly

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58067/#review170639
---




agents-common/src/main/java/org/apache/ranger/plugin/client/HadoopConfigHolder.java
Lines 76 (patched)
<https://reviews.apache.org/r/58067/#comment243517>

Please move this along with other boolean variable declaration as coding 
practice.

Do this for other class also where you declared similar boolean variables.



agents-common/src/main/java/org/apache/ranger/plugin/client/HadoopConfigHolder.java
Lines 306 (patched)
<https://reviews.apache.org/r/58067/#comment243515>

Do we need this to be set to null explicitly? Null can be its default value 
when "enableHiveMetastoreLookup" is false.



hive-agent/src/main/java/org/apache/ranger/services/hive/client/HiveClient.java
Lines 602 (patched)
<https://reviews.apache.org/r/58067/#comment243511>

This Debug message can have the "file path"? please review other similar 
error message have "filepath" in this debug message.


- Ramesh Mani


On March 30, 2017, 3:32 a.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58067/
> ---
> 
> (Updated March 30, 2017, 3:32 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1483
> https://issues.apache.org/jira/browse/RANGER-1483
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently, ranger hive service definition uses hiveserver2(hs2) jdbc driver 
> to fetch hive db/table info, which is used to populate drop downs while 
> creating/updating policies. Adding ranger hive service definition to also use 
> hive metastore client which read from hive metastore db.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/client/HadoopConfigHolder.java
>  00374de 
>   
> hive-agent/src/main/java/org/apache/ranger/services/hive/client/HiveClient.java
>  734c8e7 
>   security-admin/pom.xml 77f2b25 
>   security-admin/src/main/webapp/scripts/views/service/ConfigurationList.js 
> dcc85ab 
> 
> 
> Diff: https://reviews.apache.org/r/58067/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Tested on Simple environment.
> 2. Tested on Secure cluster
> 3. Tested with SSL
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

Re: Question about group policies

[Ramesh Mani]

Alex,

What I  was mentioning when you do plugin.isAccessAllowed(request), your
request should contain both user / group  and to get the group information
you can use Hadoop UserGroupInformation API.

That is what Don Bosco Durai was mention in this last email.

Additional comments I have put against your questions below.

Thanks,
Ramesh



On 3/29/17, 3:14 PM, "Alexander Denissov"  wrote:

>Don, Ramesh, Abhay -- thank you for your replies.
>
>I am still quite confused, though :( While Ramesh and Abhay state that a
>client needs to provide group membership explicitly when calling
>isAccessAllowed() plugin API, Don implies that it is not necessary and we
>can only call with a username.
>
>Also, one of our engineers tested with LDAP groups and says that LDAP
>groups work, while groups created via Ranger UI do not. By "work" I mean
>when a user is a member of the group and only a group policy is defined,
>then passing only the username results in policy evaluating correctly and
>granting access to the resource. I have not yet tested this LDAP scenario
>myself.
Ramesh Mani: Groups created via Ranger must be the groups which are in 
OS
or in LDAP.
 Users and Groups that are in Ranger are only for Policy 
creation
and Login into Ranger admin.
>
>So, I'll try asking again:
>- Does the client have to pass user groups to API call or passing just a
>username is sufficient ?
Ramesh Mani: 
For  plugin.isAccessAllowed(request), request should have user 
and
group. Prior to building the request you will need to make the hadoop api
call to create the UGI and create the user and group and use it in the
request.
Refer this : 
https://github.com/apache/ranger/blob/master/plugin-kafka/src/main/java/org
/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java#L1
39
Here plugin provides only the username to create UGI.
Hadoop user group mapping should be done correctly to get this user /
group mapping to be resolved . Also can make sure by doing “hdfs groups
” resolves to get the groups for that user for the user you are
doing.

Refer this parameter for more details:

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/co
re-default.xml
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback


>- If Ranger plugin is able to get user-group membership from Ranger
>Admin, does it happen during policy sync or as a separate process ? If
>separate, how often does the sync happen ?

Ramesh Mani:
Ranger Plugin gets only the policies defined in the ranger admin
periodically ( every 30 sec by default)
User and Groups are determined by the above mentioned hadoop 
api in the
plugin when request is created.

>- Might passing an empty set for roles parameter to the API circumvent
>automatic lookup (if such even exists) ? Should we pass null instead ?
Ramesh Mani :
Not sure which api you are mentioning here? 
>- Might there be a difference between handling LDAP groups vs. groups
>manually created ?
Ramesh Mani.
The core-site.xml has set of param for LDAP user group mapping. 
Or other
methods to use SSSD / Centrify / NSLCD / Winbind  to connect linux OS with
LDAP. 
First you can try with  Linux OS level user group mapping.
>--
>Thanks,
>Alex.
>
>On 2017-03-24 16:12 (-0700), Don Bosco Durai  wrote:
>> Alex
>> 
>> Both Abhay and Ramesh are correct. In the Hadoop eco-system we want to
>>ensure that the users and groups are consistent across all components.
>>And generally, AD/LDAP or Unix system user/groups are the source of
>>truth.
>> 
>> 
>> >>This also means user <--> group mapping in Ranger is NOT the
>>source of
>> >>truth, but rather a mirror of some other authentication system
>>(OS, LDAP,
>> >>etc) and a service will need to fetch this information upon user
>> >>authentication and provide to Ranger ?
>> 
>> Based on some of the earlier discussion on the HAWQ/Ranger integration
>>design model, it was decided to run the Ranger Plugin in another
>>process. If it is still the case, you just need to send the user from
>>the HAWQ side and the Ranger Plugin would be able to get the groups from
>>the AD/LDAP/Unix using Hadoop Common APIs. Ranger does the same for
>>Kafka and Solr integrations, because both these systems call the Ranger
>>plugin API only with the username.
>> 
>> Let me know if you need additional information.
>> 
>> Thanks
>> 
>> Bosco
>> 
>> 
>> 
>> 
>> On 3/24/17, 12:50 PM, 

Re: Review Request 58067: Ranger hive service definition to use hive metastore directly

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58067/#review170738
---


Ship it!




Ship It!

- Ramesh Mani


On March 31, 2017, 9:15 a.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58067/
> ---
> 
> (Updated March 31, 2017, 9:15 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1483
> https://issues.apache.org/jira/browse/RANGER-1483
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently, ranger hive service definition uses hiveserver2(hs2) jdbc driver 
> to fetch hive db/table info, which is used to populate drop downs while 
> creating/updating policies. Adding ranger hive service definition to also use 
> hive metastore client which read from hive metastore db.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/client/HadoopConfigHolder.java
>  00374de 
>   
> hive-agent/src/main/java/org/apache/ranger/services/hive/client/HiveClient.java
>  734c8e7 
>   security-admin/pom.xml 77f2b25 
>   security-admin/src/main/webapp/scripts/views/service/ConfigurationList.js 
> dcc85ab 
> 
> 
> Diff: https://reviews.apache.org/r/58067/diff/2/
> 
> 
> Testing
> ---
> 
> 1. Tested on Simple environment.
> 2. Tested on Secure cluster
> 3. Tested with SSL
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

Re: Review Request 57988: RANGER-1479 : Fix bug in ranger-*-(security/audit).xml loading, which prevented loading from JAR's

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57988/#review170739
---


Ship it!




Ship It!

- Ramesh Mani


On March 31, 2017, 12:32 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57988/
> ---
> 
> (Updated March 31, 2017, 12:32 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1479
> https://issues.apache.org/jira/browse/RANGER-1479
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RangerConfiguration first checks, if the necessary xml files are on the 
> classpath, and after it tries to convert it to a file path, check that file 
> is on the filesystem, and convert that path to back to URL. If the 
> configurations are inside a jar, then this will fail very badly (the resource 
> url will be something file://config.jar!/ranger-services.xml )
> 
> If the conversion is removed, it would work pretty nicely
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/hadoop/config/RangerConfiguration.java
>  d81f6b9 
> 
> 
> Diff: https://reviews.apache.org/r/57988/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Re: Review Request 58067: Ranger hive service definition to use hive metastore directly

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58067/#review170742
---


Ship it!




Ship It!

- Ramesh Mani


On March 31, 2017, 9:15 a.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58067/
> ---
> 
> (Updated March 31, 2017, 9:15 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1483
> https://issues.apache.org/jira/browse/RANGER-1483
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently, ranger hive service definition uses hiveserver2(hs2) jdbc driver 
> to fetch hive db/table info, which is used to populate drop downs while 
> creating/updating policies. Adding ranger hive service definition to also use 
> hive metastore client which read from hive metastore db.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/client/HadoopConfigHolder.java
>  00374de 
>   
> hive-agent/src/main/java/org/apache/ranger/services/hive/client/HiveClient.java
>  734c8e7 
>   security-admin/pom.xml 77f2b25 
>   security-admin/src/main/webapp/scripts/views/service/ConfigurationList.js 
> dcc85ab 
> 
> 
> Diff: https://reviews.apache.org/r/58067/diff/2/
> 
> 
> Testing
> ---
> 
> 1. Tested on Simple environment.
> 2. Tested on Secure cluster
> 3. Tested with SSL
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

Re: Review Request 58115: Ranger-1489: Solr plugin fails to get client address

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58115/#review170818
---


Ship it!




Ship It!

- Ramesh Mani


On March 31, 2017, 6:20 p.m., Yan Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58115/
> ---
> 
> (Updated March 31, 2017, 6:20 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> An immediate consequence is that IP-range conditions all fail for Solr 
> authorizations.
> 
> 
> Diffs
> -
> 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>  832302c 
> 
> 
> Diff: https://reviews.apache.org/r/58115/diff/1/
> 
> 
> Testing
> ---
> 
> auto and manual
> 
> 
> Thanks,
> 
> Yan Zhou
> 
>

Review Request 58165: RANGER-1495: Good coding practices recommendation by static code analysis

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58165/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Bugs: RANGER-1495
https://issues.apache.org/jira/browse/RANGER-1495


Repository: ranger


Description
---

RANGER-1495: Good coding practices recommendation by static code analysis


Diffs
-

  
hive-agent/src/main/java/org/apache/ranger/services/hive/client/HiveClient.java 
6cc62a7 


Diff: https://reviews.apache.org/r/58165/diff/1/


Testing
---

Testing done in VM


Thanks,

Ramesh Mani

Re: Review Request 58184: [RANGER-1497]Improvement of unit test coverage for ranger

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58184/#review171156
---




security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java
Lines 839 (patched)
<https://reviews.apache.org/r/58184/#comment244040>

Please check that if there is exception in serviceREST.grantAccess is it to 
be considered as failure, instead of doing e.printStackTrace()
serviceREST.grantPermission() uses serviceREST.grantAccess() for all the 
grant access and if there is a failure there that means grant failed.

Please check all the other occurances like this for handling the exception 
cases.


- Ramesh Mani


On April 5, 2017, 9:08 a.m., deepak sharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58184/
> ---
> 
> (Updated April 5, 2017, 9:08 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha and Mehul Parikh.
> 
> 
> Bugs: RANGER-1497
> https://issues.apache.org/jira/browse/RANGER-1497
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> [RANGER-1497]Improvement of unit test coverage for ranger by adding the unit 
> tests for org.apache.ranger.rest.AssetREST
> 
> 
> Diffs
> -
> 
>   security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58184/diff/2/
> 
> 
> Testing
> ---
> 
> tested the Junit tests added locally.
> 
> 
> Thanks,
> 
> deepak sharma
> 
>

Re: ranger for cassandra

[Ramesh Mani]

Earlier I have reviewed briefly  the Cassandra authorizer and it is RBAC based 
authorization model which is not a straight forward fit into Ranger's Attribute 
Based Access Control model.

Including dev list also.

Pinging Bosco / Madhan to give their thoughts.

Thanks,
Ramesh

From: anurag gujral mailto:anurag.guj...@gmail.com>>
Reply-To: "u...@ranger.apache.org" 
mailto:u...@ranger.apache.org>>
Date: Wednesday, April 19, 2017 at 3:31 PM
To: "u...@ranger.apache.org" 
mailto:u...@ranger.apache.org>>
Subject: ranger for cassandra

Hi All,
Can you please share if there is any plan to support apache ranger for 
cassandra?
Thanks,
Anurag

Re: Review Request 58229: RANGER-1481 : Capture cluster name in ranger audit info

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58229/#review172563
---


Ship it!




Ship It!

- Ramesh Mani


On April 6, 2017, 7:16 a.m., bhavik patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58229/
> ---
> 
> (Updated April 6, 2017, 7:16 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Gautam Borad, Abhay Kulkarni, Madhan 
> Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1481
> https://issues.apache.org/jira/browse/RANGER-1481
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> In order to support Ranger authorization from multiple clusters, it will be 
> useful to capture details of Ambari cluster name, Ranger needs to make 
> provision to capture that info to be shown in Audit Access logs. 
> This will be helpful when centralized Ranger is used to authorize hadoop 
> components across multiple clusters setup by Ambari.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java
>  e689e5d 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/model/AuthzAuditEvent.java 
> b547c43 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/solr/SolrAuditProvider.java
>  22aebb5 
>   
> agents-common/src/main/java/org/apache/ranger/admin/client/RangerAdminRESTClient.java
>  dec649d 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java
>  cee46a3 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequest.java
>  0668d57 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestImpl.java
>  15e872a 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestReadOnly.java
>  a18e8bc 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
>  acf8d15 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/GrantRevokeRequest.java
>  33f1dd4 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTUtils.java
>  609f717 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java
>  dedbe1e 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>  8ee3580 
>   
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>  460c692 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAccessRequest.java
>  b9f1cde 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  2baa97b 
>   
> knox-agent/src/main/java/org/apache/ranger/admin/client/RangerAdminJersey2RESTClient.java
>  fb92616 
>   
> knox-agent/src/main/java/org/apache/ranger/authorization/knox/KnoxRangerPlugin.java
>  61604b0 
>   
> knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
>  55ebf58 
>   
> plugin-atlas/src/main/java/org/apache/ranger/authorization/atlas/authorizer/RangerAtlasAuthorizer.java
>  2038645 
>   
> plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
>  472b734 
>   
> plugin-kms/src/main/java/org/apache/ranger/authorization/kms/authorizer/RangerKmsAuthorizer.java
>  9bebafa 
>   
> plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  2338ba1 
>   security-admin/db/mysql/patches/026-add-column-in-x_policy_export_audit.sql 
> PRE-CREATION 
>   
> security-admin/db/oracle/patches/026-add-column-in-x_policy_export_audit.sql 
> PRE-CREATION 
>   
> security-admin/db/postgres/patches/026-add-column-in-x_policy_export_audit.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlanywhere/patches/026-add-column-in-x_policy_export_audit.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlserver/patches/026-add-column-in-x_policy_export_audit.sql
>  PRE-CREATION 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyExportAudit.java
>  4544614 
>   security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java f0d2401 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> b9f1832 
&g

Review Request 58658: RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58658/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Bugs: RANGER-1513
https://issues.apache.org/jira/browse/RANGER-1513


Repository: ranger


Description
---

RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin


Diffs
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-hive.json 
b254d20 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 2baa97b 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveResource.java
 09ecd1e 
  
hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
 57b4eef 
  hive-agent/src/test/resources/hive-policies.json 2b568dc 


Diff: https://reviews.apache.org/r/58658/diff/1/


Testing
---

Test in local VM


Thanks,

Ramesh Mani

Re: Review Request 58658: RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58658/
---

(Updated April 30, 2017, 9:37 a.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Changes
---

upgrade scenario handling


Bugs: RANGER-1513
https://issues.apache.org/jira/browse/RANGER-1513


Repository: ranger


Description
---

RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin


Diffs (updated)
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-hive.json 
b254d20 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 2baa97b 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveResource.java
 09ecd1e 
  
hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
 57b4eef 
  hive-agent/src/test/resources/hive-policies.json 2b568dc 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchForHiveServiceDefUpdate_J10006.java
 7d6a23d 


Diff: https://reviews.apache.org/r/58658/diff/2/

Changes: https://reviews.apache.org/r/58658/diff/1-2/


Testing
---

Test in local VM


Thanks,

Ramesh Mani

Re: Review Request 58658: RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58658/
---

(Updated May 4, 2017, 5:10 p.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Changes
---

Fixed Review comments


Bugs: RANGER-1513
https://issues.apache.org/jira/browse/RANGER-1513


Repository: ranger


Description
---

RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin


Diffs (updated)
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-hive.json 
b254d20 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 2baa97b 
  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveResource.java
 09ecd1e 
  
hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
 57b4eef 
  hive-agent/src/test/resources/hive-policies.json 2b568dc 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchForHiveServiceDefUpdate_J10007.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/58658/diff/3/

Changes: https://reviews.apache.org/r/58658/diff/2-3/


Testing
---

Test in local VM


Thanks,

Ramesh Mani

Re: Review Request 58658: RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin

[Ramesh Mani]

> On April 30, 2017, 8:25 p.m., Madhan Neethiraj wrote:
> > hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
> > Lines 1415 (patched)
> > <https://reviews.apache.org/r/58658/diff/2/?file=1703904#file1703904line1416>
> >
> > Would current URI access checks (for hdfs://, file://), that are 
> > performed by FileSystem, be done using Ranger Hive policies after this 
> > patch?
> > 
> > If yes, new Ranger Hive policies would have to be added to allow URL 
> > access; else it might break existing deployments. Please review.
> > 
> > One option to consider is to fallback on FileSystem access check when 
> > there is no Ranger Hive policy to grant the necessary access.

Currently URI check for hdfs://, file:// are not done by the Hive Policies, it 
is be done by the FileSystem access check. Only if its s3a://, s3n:// it will 
be done by Hive Policy.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58658/#review173456
-------


On May 4, 2017, 5:10 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58658/
> ---
> 
> (Updated May 4, 2017, 5:10 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-1513
> https://issues.apache.org/jira/browse/RANGER-1513
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1513:Add Support for S3 authorization in Ranger Hive Plugin
> 
> 
> Diffs
> -
> 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-hive.json 
> b254d20 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  2baa97b 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveResource.java
>  09ecd1e 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  57b4eef 
>   hive-agent/src/test/resources/hive-policies.json 2b568dc 
>   
> security-admin/src/main/java/org/apache/ranger/patch/PatchForHiveServiceDefUpdate_J10007.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58658/diff/3/
> 
> 
> Testing
> ---
> 
> Test in local VM
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Review Request 59041: RANGER-1561:Good coding practice in Ranger recommended by static code analysis

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59041/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Bugs: RANGER-1561
https://issues.apache.org/jira/browse/RANGER-1561


Repository: ranger


Description
---

RANGER-1561:Good coding practice in Ranger recommended by static code analysis


Diffs
-

  
hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
 c4c2a2a 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchForHiveServiceDefUpdate_J10007.java
 a886945 


Diff: https://reviews.apache.org/r/59041/diff/1/


Testing
---

Tested locally


Thanks,

Ramesh Mani

Review Request 59245: RANGER-1577:Update Ranger-WASB servicedefinition to remove Execute permission and disallow policies with a trailing slash

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59245/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Repository: ranger


Description
---

RANGER-1577:Update Ranger-WASB servicedefinition to remove Execute permission 
and disallow policies with a trailing slash


Diffs
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-wasb.json 
038ebaf 


Diff: https://reviews.apache.org/r/59245/diff/1/


Testing
---


Thanks,

Ramesh Mani

Re: [VOTE] Release Apache Ranger 0.7.1 - release candidate 1 (dev group vote)

[Ramesh Mani]

+1

- Did successful build of apache-ranger-0.7.1.tar.gz  with rat plugin.
- Verified some source files.
- Verified PGP signature, MD5/SHA hash.

Thanks,
Ramesh





On 6/1/17, 8:08 AM, "Velmurugan Periasamy"  wrote:

>Hello Rangers:
>
>Apache Ranger 0.7.1 release candidate #1 is now available for a vote
>within dev community. Links to release artifacts are given below. Could
>you please review and vote?
>
>The vote will be open for at least 72 hours or until necessary number of
>votes are reached.
>[ ] +1  approve
>[ ] +0  no opinion
>[ ] -1  disapprove (and reason why)
>
>Here is my +1 (binding)
>
>Thank you,
>Vel
>
>Git tag for the release:
>  https://github.com/apache/ranger/tree/ranger-0.7.1-rc1  (last commit id
>:  098865edd9174ab5bb17bf5c7caa54128fdf04d2)
>Sources for the release:
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.7.1-rc1/apache-ranger-0.7.
>1.tar.gz
>
>Source release verification:
>PGP Signature:  
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.7.1-rc1/apache-ranger-0.7.
>1.tar.gz.asc 
>MD5/SHA  Hash:
>  
>https://dist.apache.org/repos/dist/dev/ranger/0.7.1-rc1/apache-ranger-0.7.
>1.tar.gz.mds 
>
>Keys to verify the signature of the release artifact are available at:
>  https://people.apache.org/keys/group/ranger.asc
>
>

Re: Review Request 59753: RANGER-1630 - StormClient doesn't decrypt password

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59753/#review176822
---




storm-agent/src/main/java/org/apache/ranger/services/storm/client/StormClient.java
Lines 399 (patched)
<https://reviews.apache.org/r/59753/#comment250301>

Minor error  "Storm" connection not knox connection.


- Ramesh Mani


On June 2, 2017, 4:46 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59753/
> ---
> 
> (Updated June 2, 2017, 4:46 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1630
> https://issues.apache.org/jira/browse/RANGER-1630
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The StormClient doesn't decrypt the password. That means if you create a 
> service, save it, and go back and edit it and test the connection, it will 
> fail. I also did some refactoring to move the local class out to a private 
> static class instead.
> 
> 
> Diffs
> -
> 
>   
> storm-agent/src/main/java/org/apache/ranger/services/storm/client/StormClient.java
>  b72a9a23 
> 
> 
> Diff: https://reviews.apache.org/r/59753/diff/1/
> 
> 
> Testing
> ---
> 
> Tested we can connect successfully to Storm.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 59753: RANGER-1630 - StormClient doesn't decrypt password

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59753/#review176898
---


Ship it!




fix the minor error and ship it

- Ramesh Mani


On June 2, 2017, 4:46 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59753/
> ---
> 
> (Updated June 2, 2017, 4:46 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1630
> https://issues.apache.org/jira/browse/RANGER-1630
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The StormClient doesn't decrypt the password. That means if you create a 
> service, save it, and go back and edit it and test the connection, it will 
> fail. I also did some refactoring to move the local class out to a private 
> static class instead.
> 
> 
> Diffs
> -
> 
>   
> storm-agent/src/main/java/org/apache/ranger/services/storm/client/StormClient.java
>  b72a9a23 
> 
> 
> Diff: https://reviews.apache.org/r/59753/diff/1/
> 
> 
> Testing
> ---
> 
> Tested we can connect successfully to Storm.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Review Request 59809: RANGER-1631 : create temp function failing with permission issues

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59809/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Repository: ranger


Description
---

RANGER-1631 : create temp function failing with permission issues


Diffs
-

  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveResource.java
 3f1279f 


Diff: https://reviews.apache.org/r/59809/diff/1/


Testing
---

Testing done in local VM with command 

create temporary function row_sequence as 
'org.apache.hadoop.hive.contrib.udf.UDFRowSequence' using jar 
'/usr/hdp/2.6.1.0-122/hive/lib/hive-contrib-1.2.1000.2.6.1.0-122.jar';


Thanks,

Ramesh Mani

Review Request 60167: RANGER-1648: Ranger Kafka Plugin now should use the Short name from Kafka Session Object

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60167/
---

Review request for ranger, Madhan Neethiraj, Selvamohan Neethiraj, and 
Velmurugan Periasamy.


Bugs: RANGER-1648
https://issues.apache.org/jira/browse/RANGER-1648


Repository: ranger


Description
---

RANGER-1648: Ranger Kafka Plugin now should use the Short name from Kafka 
Session Object


Diffs
-

  
plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
 ec7f887 


Diff: https://reviews.apache.org/r/60167/diff/1/


Testing
---

Testing in local vm with kafka plugin.


Thanks,

Ramesh Mani

Review Request 60256: RANGER-1658: Solr gives NPE while printing the AuthorizationContext in INFO and DEBUG log

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60256/
---

Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy.


Bugs: RANGER-1658
https://issues.apache.org/jira/browse/RANGER-1658


Repository: ranger


Description
---

RANGER-1658: Solr gives NPE while printing the AuthorizationContext in INFO and 
DEBUG log


Diffs
-

  
plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
 bf22b47 


Diff: https://reviews.apache.org/r/60256/diff/1/


Testing
---

Testing done with Local VM


Thanks,

Ramesh Mani

Re: Review Request 60361: RANGER-1663 - Make Kafka GSS test more robust

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60361/#review178802
---


Ship it!




Ship It!

- Ramesh Mani


On June 23, 2017, 9:02 a.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60361/
> ---
> 
> (Updated June 23, 2017, 9:02 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1663
> https://issues.apache.org/jira/browse/RANGER-1663
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The Kafka GSS tests seem to fail on some Jenkins servers. I think this might 
> be due to the hard-coded dependency on "127.0.0.1" for Zookeeper.
> 
> 
> Diffs
> -
> 
>   
> plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java
>  2b65cbf8 
>   plugin-kafka/src/test/resources/kafka_kerberos.jaas b764932d 
> 
> 
> Diff: https://reviews.apache.org/r/60361/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 60454: RANGER-1669:We need to support the original functionality of hive.show grant user username

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60454/#review179001
---




hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizerBase.java
Lines 67 (patched)
<https://reviews.apache.org/r/60454/#comment253383>

Falling back to Native SQLStdHiveAccessControler is not a ideal way to 
resolve this. HiveAuthorizer should provide the necessary implementation for 
showPrivileges(). This has to be discussed with Hive comitter also.


- Ramesh Mani


On June 27, 2017, 10:57 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60454/
> ---
> 
> (Updated June 27, 2017, 10:57 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, 
> Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1669
> https://issues.apache.org/jira/browse/RANGER-1669
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> We need to support the original functionality of hive,
> 1.Scene: the original hive supports sql:show grant user glc;
> 2.Error: jdbc:hive2://10.43.182.241:1> show grant user glc;
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. showPrivileges() not 
> implemented in Ranger HiveAuthorizer (state=08S01,code=1)
> 3.Solution: When you integrate the Ranger plug-in into hive, you can't 
> execute show grant user glc.
> Modify RangerHiveAuthorizerBase.java to achieve the specific implementation 
> of showPrivileges,
> 4.The modified results can continue to support the original functionality of 
> the hive.
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizerBase.java
>  c313870 
> 
> 
> Diff: https://reviews.apache.org/r/60454/diff/1/
> 
> 
> Testing
> ---
> 
> tested it
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 60454: RANGER-1669:We need to support the original functionality of hive.show grant user username

[Ramesh Mani]

> On June 27, 2017, 2:46 p.m., Qiang Zhang wrote:
> > Ship It!

Qiang Zhan, Please don't commit this in Apache as this might need more 
discussion that a patch up fix.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60454/#review178983
---


On June 27, 2017, 10:57 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60454/
> ---
> 
> (Updated June 27, 2017, 10:57 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, 
> Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1669
> https://issues.apache.org/jira/browse/RANGER-1669
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> We need to support the original functionality of hive,
> 1.Scene: the original hive supports sql:show grant user glc;
> 2.Error: jdbc:hive2://10.43.182.241:1> show grant user glc;
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. showPrivileges() not 
> implemented in Ranger HiveAuthorizer (state=08S01,code=1)
> 3.Solution: When you integrate the Ranger plug-in into hive, you can't 
> execute show grant user glc.
> Modify RangerHiveAuthorizerBase.java to achieve the specific implementation 
> of showPrivileges,
> 4.The modified results can continue to support the original functionality of 
> the hive.
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizerBase.java
>  c313870 
> 
> 
> Diff: https://reviews.apache.org/r/60454/diff/1/
> 
> 
> Testing
> ---
> 
> tested it
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 60596: Ranger UI should consider "recursiveSupported" attribute value at each resource level to Store the Policy.

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60596/#review179904
---


Ship it!




Ship It!

- Ramesh Mani


On July 4, 2017, 5:44 a.m., Nitin Galave wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60596/
> ---
> 
> (Updated July 4, 2017, 5:44 a.m.)
> 
> 
> Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, 
> Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1666
> https://issues.apache.org/jira/browse/RANGER-1666
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger UI should consider "recursiveSupported" value at each resource level 
> to Store the Policy. Currently, if there is resource matcher with 
> "recursiveSupported" = true and others resource has false, UI call the 
> createPolicy REST with "true" for all resources in the policy which resulted 
> in the failure of the policy saving.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/webapp/scripts/modules/XAOverrides.js 7d7a9d1 
>   security-admin/src/main/webapp/scripts/views/policies/RangerPolicyForm.js 
> 9145b88 
>   security-admin/src/main/webapp/styles/xa.css a1037b9 
> 
> 
> Diff: https://reviews.apache.org/r/60596/diff/1/
> 
> 
> Testing
> ---
> 
> Verified : 
> 1. Showing resource level recusive toggle button in the policy screen.
> 2. Policy CRUD for hdfs, hive, yarn component.
> 
> 
> Thanks,
> 
> Nitin Galave
> 
>

Review Request 60781: RANGER-1689:Add support for defining recursive policies for WASB service def

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60781/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Repository: ranger


Description
---

RANGER-1689:Add support for defining recursive policies for WASB service def


Diffs
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-wasb.json 
9b3eafe 


Diff: https://reviews.apache.org/r/60781/diff/1/


Testing
---

Test in local VM


Thanks,

Ramesh Mani

Re: Review Request 61021: RANGER-1672:Ranger supports plugin to enable, monitor and manage apache kylin

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61021/#review181282
---




agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json
Lines 18 (patched)
<https://reviews.apache.org/r/61021/#comment256808>

Is this going to be "String" to be Matched. If so use 
org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher. Refer 
ranger-servicedef-hive.json for it.



plugin-kylin/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
Lines 71 (patched)
<https://reviews.apache.org/r/61021/#comment256810>

Why there is a check for "Cube" first and then "Project". Is there a 
resource hierarchy like Project -> cubes. If so does the "Project" has to be a 
another resource in the resource definition? Please review this and update. 
Refer Hive Plugin if needed.



security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
Line 2764 (original), 2769 (patched)
<https://reviews.apache.org/r/61021/#comment256812>

User/Group and default policy should be moved to 
org.apache.ranger.services.hdfs.RangerServiceKylin. You need to create one and 
refer it in ranger-servicedef-kylin.json. 
ServiceDBStore.java should have call only to createDefaultPolicy(). This 
class should not be changed as such.

 Refer HDFS/KMS plugin and getDefaultRangerPolicies for those plugin in 
org.apache.ranger.services.hdfs.RangerService



security-admin/src/main/java/org/apache/ranger/biz/ServiceMgr.java
Lines 581 (patched)
<https://reviews.apache.org/r/61021/#comment256815>

searchKylinUsers should be done in SearchUtil.java. THis class as such 
cannot be changed. It is only for Service related calls. Please move this out 
and place it in SearchUtil. Look at other calls in XUserREST.java



security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java
Lines 372 (patched)
<https://reviews.apache.org/r/61021/#comment256818>

    Repeated class kylinUserResponse. Make it part of Util class


- Ramesh Mani


On July 21, 2017, 7:40 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61021/
> ---
> 
> (Updated July 21, 2017, 7:40 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1672
> https://issues.apache.org/jira/browse/RANGER-1672
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger supports plugin to enable, monitor and manage apache kylin
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  0bc09f6 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json 
> PRE-CREATION 
>   plugin-kylin/.gitignore PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-kylin/pom.xml PRE-CREATION 
>   plugin-kylin/scripts/install.properties PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/RangerServiceKylin.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinClient.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinConnectionMgr.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinResourceMgr.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinCubeResponse.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinProjectResponse.java
>  PRE-CREATION 
>   pom.xml 3fcc4aa 
>   ranger-kylin-plugin-shim/.gitignore PRE-CREATION 
>   ranger-kylin-plu

Re: Review Request 61021: RANGER-1672:Ranger supports plugin to enable, monitor and manage apache kylin

[Ramesh Mani]

> On July 24, 2017, 11:17 p.m., Ramesh Mani wrote:
> >

This seems to be interesting contribution! Thanks.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61021/#review181282
---


On July 21, 2017, 7:40 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61021/
> ---
> 
> (Updated July 21, 2017, 7:40 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1672
> https://issues.apache.org/jira/browse/RANGER-1672
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger supports plugin to enable, monitor and manage apache kylin
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  0bc09f6 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json 
> PRE-CREATION 
>   plugin-kylin/.gitignore PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-kylin/pom.xml PRE-CREATION 
>   plugin-kylin/scripts/install.properties PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/RangerServiceKylin.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinClient.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinConnectionMgr.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinResourceMgr.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinCubeResponse.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinProjectResponse.java
>  PRE-CREATION 
>   pom.xml 3fcc4aa 
>   ranger-kylin-plugin-shim/.gitignore PRE-CREATION 
>   ranger-kylin-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-kylin-plugin-shim/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 9742265 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceMgr.java 722a566 
>   security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 6e9161e 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> af7bdfe 
>   security-admin/src/main/webapp/scripts/views/policies/PermissionList.js 
> 067bf3b 
>   security-admin/src/main/webapp/scripts/views/policies/RangerPolicyForm.js 
> 6f27d5d 
>   src/main/assembly/admin-web.xml cb1aad2 
>   src/main/assembly/plugin-kylin.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61021/diff/1/
> 
> 
> Testing
> ---
> 
> Tested
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Re: Review Request 58272: Ranger-1502: Solr shutdown does not cause the audit log file to be flushed and closed

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58272/#review182129
---


Ship it!




Ship It!

- Ramesh Mani


On April 7, 2017, 9:48 p.m., Yan Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58272/
> ---
> 
> (Updated April 7, 2017, 9:48 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The current audit stream close mechanism uses a Java shutdown hook registered 
> with Hadoop's ShutdownHookManager. Solr shutdown, however, somehow does not 
> cause the shutdown hook to be invoked, potentially resulting lost audit logs. 
> We are experiencing lost logs toward HDFS audit destination.
> 
> The fix is to add invocation of the shutdown hook from the close() call of 
> the Solr's AuthorizationPlugin interface, in addition to the existing 
> invocation mechanism from the JVM invocation of shutdown hooks.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditProviderFactory.java
>  b37011e6 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>  bf22b476 
> 
> 
> Diff: https://reviews.apache.org/r/58272/diff/1/
> 
> 
> Testing
> ---
> 
> automated and manual
> 
> 
> Thanks,
> 
> Yan Zhou
> 
>

Re: Review Request 58268: Ranger-1501: Audit Flush to HDFS does not actually cause the audit logs to be flushed to HDFS

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58268/#review182130
---


Ship it!




Ship It!

- Ramesh Mani


On April 25, 2017, 2:54 p.m., Yan Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58268/
> ---
> 
> (Updated April 25, 2017, 2:54 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1501
> https://issues.apache.org/jira/browse/RANGER-1501
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The reason is that HDFS file stream's flush() call does not really flush the 
> data all the way to disk, nor even makes the data visible to HDFS users. See 
> the HDFS semantics of the flush/sync at 
> https://issues.apache.org/jira/browse/HADOOP-6313.
> Consequently the audit logs on HDFS won't be visible/durable from HDFS client 
> until the log file is closed. This will, among other issues, boost chances of 
> losing audit logs in case of system failure.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
>  889b6ffd 
> 
> 
> Diff: https://reviews.apache.org/r/58268/diff/1/
> 
> 
> Testing
> ---
> 
> Automated and manual
> 
> 
> Thanks,
> 
> Yan Zhou
> 
>

Review Request 61412: RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket renewal mechanism

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61412/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Selvamohan 
Neethiraj, and Velmurugan Periasamy.


Bugs: RANGER-1649
https://issues.apache.org/jira/browse/RANGER-1649


Repository: ranger


Description
---

RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket 
renewal mechanism


Diffs
-

  
plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
 5c4e066 


Diff: https://reviews.apache.org/r/61412/diff/1/


Testing
---

Testing done in local VM.


Thanks,

Ramesh Mani

Review Request 61419: RANGER-1501:Audit Flush to HDFS does not actually cause the audit logs to be flushed to HDFS- improvement patch

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61419/
---

Review request for ranger, Abhay Kulkarni and Madhan Neethiraj.


Bugs: RANGER-1501
https://issues.apache.org/jira/browse/RANGER-1501


Repository: ranger


Description
---

RANGER-1501:Audit Flush to HDFS does not actually cause the audit logs to be 
flushed to HDFS- improvement patch


Diffs
-

  
agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
 1a15c30 


Diff: https://reviews.apache.org/r/61419/diff/1/


Testing
---

Tested in local vm


Thanks,

Ramesh Mani

Re: Review Request 60781: RANGER-1689:Add support for defining recursive policies for WASB service def

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60781/
---

(Updated Aug. 4, 2017, 7:55 a.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Bugs: RANGER-1689
https://issues.apache.org/jira/browse/RANGER-1689


Repository: ranger


Description
---

RANGER-1689:Add support for defining recursive policies for WASB service def


Diffs
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-wasb.json 
9b3eafe 


Diff: https://reviews.apache.org/r/60781/diff/1/


Testing
---

Test in local VM


Thanks,

Ramesh Mani

Re: Review Request 61412: RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket renewal mechanism

[Ramesh Mani]

> On Aug. 4, 2017, 10:23 a.m., Colm O hEigeartaigh wrote:
> > Why does setUGIFromJAASConfig solve the problem as opposed to 
> > authWithConfig? It's not really clear from the bug description. One 
> > potential issue is that setUGIFromJAASConfig requires a KeyTab in JAAS 
> > configuration, whereas authWithConfig looks like it would work with a 
> > password.

Your Observation is correct. Ranger Plugin for non core Hadoop components like 
Solr, when it uses Hadoop UserGroupInformation api to set/get the UGI, and this 
UGI is used for Authenticated call to Download Policy / Audit to HDFS. When TGT 
expires there was failure as it never got renewed. (Core components like Hdfs, 
hive, hbase internally taking care of this with right keytab login and renewal 
). So in this case when we do a MiscUtil.getUGILoginUser() to get UGI at the 
plugin, this call will invoke UGI.checkTGTAndReloginFromKeytab() to check and 
renew the TGT. This fails if the UGI is not created with Principal/Keytab.
In this issue when authWithConfig(), it uses the just Subject() alone to login 
and as a result checkTGTAndReloginFromKeytab() failed. I have updated the 
Description with the details.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61412/#review182195
---


On Aug. 3, 2017, 6:53 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61412/
> ---
> 
> (Updated Aug. 3, 2017, 6:53 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Selvamohan 
> Neethiraj, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1649
> https://issues.apache.org/jira/browse/RANGER-1649
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in 
> ticket renewal mechanism
> 
> 
> Diffs
> -
> 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>  5c4e066 
> 
> 
> Diff: https://reviews.apache.org/r/61412/diff/1/
> 
> 
> Testing
> ---
> 
> Testing done in local VM.
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: New Defects reported by Coverity Scan for Apache Ranger

[Ramesh Mani]

Bosco,

I am working on these defects,

Thanks,
Ramesh

On 8/6/17, 1:10 PM, "Don Bosco Durai"  wrote:

>Please review and fix.
>
>Thanks
>
>Bosco
>
>
>On 8/6/17, 12:44 AM, "scan-ad...@coverity.com" 
>wrote:
>
>
>Hi,
>
>Please find the latest report on new defect(s) introduced to Apache
>Ranger found with Coverity Scan.
>
>2 new defect(s) introduced to Apache Ranger found with Coverity Scan.
>
>
>New defect(s) Reported-by: Coverity Scan
>Showing 2 of 2 defect(s)
>
>
>** CID 166171:  FindBugs: Multithreaded correctness
>(FB.RU_INVOKE_RUN)
>
>/agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditProvider
>Factory.java: 117 in
>org.apache.ranger.audit.provider.AuditProviderFactory.shutdown()()
>
>
>
>__
>__
>*** CID 166171:  FindBugs: Multithreaded correctness
>(FB.RU_INVOKE_RUN)
>
>/agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditProvider
>Factory.java: 117 in
>org.apache.ranger.audit.provider.AuditProviderFactory.shutdown()()
>111/**
>112 * call shutdown hook to provide a way to
>113 * shutdown gracefully in addition to the ShutdownHook
>mechanism
>114 */
>115public void shutdown() {
>116if (isInitDone() && jvmShutdownHook != null) {
>>>> CID 166171:  FindBugs: Multithreaded correctness
>(FB.RU_INVOKE_RUN)
>>>> 
>org.apache.ranger.audit.provider.AuditProviderFactory.shutdown()
>explicitly invokes run on a thread (did you mean to start it instead?).
>117jvmShutdownHook.run();
>118}
>119}
>120 
>121public synchronized void init(Properties props, String
>appType) {
>122LOG.info("AuditProviderFactory: initializing..");
>
>** CID 166170:  Control flow issues  (NESTING_INDENT_MISMATCH)
>
>/agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditD
>estination.java: 199 in
>org.apache.ranger.audit.destination.HDFSAuditDestination.flush()()
>
>
>
>__
>__
>*** CID 166170:  Control flow issues  (NESTING_INDENT_MISMATCH)
>
>/agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditD
>estination.java: 199 in
>org.apache.ranger.audit.destination.HDFSAuditDestination.flush()()
>193synchronized (this) {
>194if (ostream != null)
>195// 1) PrinterWriter 
> does not have bufferring of its own
>so
>196// we need to flush its 
> underlying stream
>197// 2) HDFS flush() does 
> not really flush all the way to
>disk.
>198ostream.hflush();
>>>> CID 166170:  Control flow issues  (NESTING_INDENT_MISMATCH)
>>>> This  statement is indented to column 49, as if it were
>nested within the preceding parent statement, but it is not.
>199logger.info("Flush HDFS 
> audit logs completed.");
>200}
>201} catch (IOException e) {
>202logger.error("Error on flushing log 
> writer: " +
>e.getMessage() +
>203 "\nException will be ignored. name=" + 
> getName() + ",
>fileName=" + currentFileName);
>204}
>
>
>
>__
>__
>To view the defects in Coverity Scan visit,
>https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V
>05UPxvVjWch-2Bd2MGckcRZSbhom32dlDl11LWEm9nX11zsOWMf5dv3Q9Mogo-2FGua3FsLRTF
>ft2V-2FOFC9o0P2e0-3D_d04ZgyDzSjlwpjXIuOFYDNE6R93Lal83MDClQK32PZv33XLds5st2
>CH16GjUjfPDIC28Lk2AlHQ2-2BKTWLfVhhV4FUtxSH-2BQ-2FYdiREYij94dL6Vnyx3h86Wdgp
>d9-2Fq10Q7jqbIroRL1-2FvMV-2FOO483ZsHqVoHPsly3MZ-2B-2F5WjaCjwhmF-2Fz5-2F2SR
>i18UKgQmkJsXC3iUEFy8HsU4Ji7c8e4TA-3D-3D
>
>To manage Coverity Scan email notifications for "bo...@apache.org",
>click 
>https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V
>05UPxvVjWch-2Bd2MGckcRbVDbis712qZDP-2FA8y06Nq4HK0JLY-2BbZ-2FD0yvjg-2BbWSwq
>uqqdEYtbR9nIDW-2BM81kI8TiABM2LsH3tiPfMWf-2FvOsjZSWngS5IRVC-2FH5Pl4zyaK1OE6
>Dh-2BhR6pXASEFJKZLM-3D_d04ZgyDzSjlwpjXIuOFYDNE6R93Lal83MDClQK32PZv33XLds5s
>t2CH16GjUjfPDu9D6-2FiICYejLSuAywfM1j4jBCjl449cBsU7cKc1B6BCx-2BDSASW9dUTiEo
>AdcTj-2FAP-2FIcCKkeRevD-2FxKiZ5t5tTOtoT7TFe9DXg3C5TeI-2FuLIaHM-2BJrzyK5rxg
>lD2SY0eVa0HwhK4xEM3-2F5x2-2FhNrZQ-3D-3D
>
>
>
>

Re: Review Request 61412: RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket renewal mechanism

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61412/
---

(Updated Aug. 8, 2017, 12:45 a.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Selvamohan 
Neethiraj, and Velmurugan Periasamy.


Changes
---

RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket 
renewal mechanism - PMD fix


Bugs: RANGER-1649
https://issues.apache.org/jira/browse/RANGER-1649


Repository: ranger


Description
---

RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket 
renewal mechanism


Diffs (updated)
-

  
plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
 5c4e066 


Diff: https://reviews.apache.org/r/61412/diff/2/

Changes: https://reviews.apache.org/r/61412/diff/1-2/


Testing (updated)
---

fixed PMD issue with earlier patch.
Testing done in local VM.


Thanks,

Ramesh Mani

Re: Review Request 61412: RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket renewal mechanism

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61412/
---

(Updated Aug. 9, 2017, 5:37 p.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Selvamohan 
Neethiraj, and Velmurugan Periasamy.


Changes
---

Remove unused function


Bugs: RANGER-1649
https://issues.apache.org/jira/browse/RANGER-1649


Repository: ranger


Description
---

RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket 
renewal mechanism


Diffs (updated)
-

  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
7a1d458 
  
plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
 5c4e066 


Diff: https://reviews.apache.org/r/61412/diff/3/

Changes: https://reviews.apache.org/r/61412/diff/2-3/


Testing
---

fixed PMD issue with earlier patch.
Testing done in local VM.


Thanks,

Ramesh Mani

Re: Review Request 61412: RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in ticket renewal mechanism

[Ramesh Mani]

> On Aug. 8, 2017, 4:37 p.m., Colm O hEigeartaigh wrote:
> > Thanks for the explanation! You could delete the "authWithConfig" method as 
> > part of this patch, as it's not used by any other code.

I have removed the unused function


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61412/#review182403
---


On Aug. 9, 2017, 5:37 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61412/
> ---
> 
> (Updated Aug. 9, 2017, 5:37 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Selvamohan 
> Neethiraj, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1649
> https://issues.apache.org/jira/browse/RANGER-1649
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1649:Ranger Solr Plugin fails to refresh policy due to failure in 
> ticket renewal mechanism
> 
> 
> Diffs
> -
> 
>   agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
> 7a1d458 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>  5c4e066 
> 
> 
> Diff: https://reviews.apache.org/r/61412/diff/3/
> 
> 
> Testing
> ---
> 
> fixed PMD issue with earlier patch.
> Testing done in local VM.
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 61594: RANGER-1731: Exclude multiple guava versions

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61594/#review182729
---


Ship it!




Ship It!

- Ramesh Mani


On Aug. 11, 2017, 1:10 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61594/
> ---
> 
> (Updated Aug. 11, 2017, 1:10 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1731
> https://issues.apache.org/jira/browse/RANGER-1731
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Guava is included in multiple places - even with different versions. The most 
> problematic one, is the owasp-java-html-sanitizer which only specifies a 
> version range. This could cause problem during build, if maven cache is empty 
> and it unable to load the necessary jars, eg. guava 11.0 - which is not 
> needed in the end at all.
> 
> 
> Diffs
> -
> 
>   security-admin/pom.xml cc39be8 
> 
> 
> Diff: https://reviews.apache.org/r/61594/diff/1/
> 
> 
> Testing
> ---
> 
> https://travis-ci.org/gzsombor/ranger/builds/263479478
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Re: Review Request 61652: RANGER-1734 - Close the connection in the HIVERangerAuthorizerTest

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61652/#review182952
---


Ship it!




Ship It!

- Ramesh Mani


On Aug. 15, 2017, 9:06 a.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61652/
> ---
> 
> (Updated Aug. 15, 2017, 9:06 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1734
> https://issues.apache.org/jira/browse/RANGER-1734
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Close the connection in the HIVERangerAuthorizerTest
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  c6f0b8f9 
> 
> 
> Diff: https://reviews.apache.org/r/61652/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 61783: created unit test for RANGER-1631

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61783/#review183542
---




hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
Lines 334 (patched)
<https://reviews.apache.org/r/61783/#comment259547>

- I am not able to apply this patch on master. please rebase it to latest 
code and update this review patch.
- Also please remove the unnecessary whitespace in the 
HIVERangerAuthorizerTest.java code which shows up in RED bars in the review.


- Ramesh Mani


On Aug. 21, 2017, 3:04 p.m., Endre Zoltan Kovacs wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61783/
> ---
> 
> (Updated Aug. 21, 2017, 3:04 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1631
> https://issues.apache.org/jira/browse/RANGER-1631
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added unit test for RANGER-1631
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  2c9e9552 
>   hive-agent/src/test/resources/hive-policies.json 41a4e203 
> 
> 
> Diff: https://reviews.apache.org/r/61783/diff/1/
> 
> 
> Testing
> ---
> 
> added unit test to cover use case of: 
> RANGER-1631: create temp function failing with permission issues 
> (2e193e124399cf685c17798b8243e1d62f223315)
> 
> My unit test creates a database test1.
> Creates a UDF for test1.
> Creates a temporary UDF (which is not bound to any database)
> Asserts for the presence of these 2 UDFs.
> 
> The test case includes: 1 Junit test method in 
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
> 
> and a new User called: tom in the policies file:
> hive-agent/src/test/resources/hive-policies.json
> 
> tom has Ranger policies to:
> - create/read/update/delete databases
> - create/read/update/delete UDF on test1 database
> 
> 
> this unit test WILL fail when commit 2e193e124399cf685c17798b8243e1d62f223315 
> is not present / reverted
> (HiveAccessControlException: Permission denied: user [tom] does not have 
> [CREATE] privilege on [tmp])
> , and it will pass when that commit is present.
> 
> 
> To run this test:
> cd ranger/hive-agent
> mvn test -Dtest=HIVERangerAuthorizerTest#testHiveUdfCreateOnWildcardDatabase
> 
> To run this test alongside with its siblings:
> cd ranger/hive-agent
> mvn test -Dtest=HIVERangerAuthorizerTest
> 
> 
> Thanks,
> 
> Endre Zoltan Kovacs
> 
>

Re: Review Request 61783: created unit test for RANGER-1631

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61783/#review184322
---


Ship it!




Ship It!

- Ramesh Mani


On Aug. 23, 2017, 10:57 a.m., Endre Zoltan Kovacs wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61783/
> ---
> 
> (Updated Aug. 23, 2017, 10:57 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1631
> https://issues.apache.org/jira/browse/RANGER-1631
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> added unit test for RANGER-1631
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  2c9e9552 
>   hive-agent/src/test/resources/hive-policies.json 41a4e203 
> 
> 
> Diff: https://reviews.apache.org/r/61783/diff/2/
> 
> 
> Testing
> ---
> 
> added unit test to cover use case of: 
> RANGER-1631: create temp function failing with permission issues 
> (2e193e124399cf685c17798b8243e1d62f223315)
> 
> My unit test creates a database test1.
> Creates a UDF for test1.
> Creates a temporary UDF (which is not bound to any database)
> Asserts for the presence of these 2 UDFs.
> 
> The test case includes: 1 Junit test method in 
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
> 
> and a new User called: tom in the policies file:
> hive-agent/src/test/resources/hive-policies.json
> 
> tom has Ranger policies to:
> - create/read/update/delete databases
> - create/read/update/delete UDF on test1 database
> 
> 
> this unit test WILL fail when commit 2e193e124399cf685c17798b8243e1d62f223315 
> is not present / reverted
> (HiveAccessControlException: Permission denied: user [tom] does not have 
> [CREATE] privilege on [tmp])
> , and it will pass when that commit is present.
> 
> 
> To run this test:
> cd ranger/hive-agent
> mvn test -Dtest=HIVERangerAuthorizerTest#testHiveUdfCreateOnWildcardDatabase
> 
> To run this test alongside with its siblings:
> cd ranger/hive-agent
> mvn test -Dtest=HIVERangerAuthorizerTest
> 
> 
> Thanks,
> 
> Endre Zoltan Kovacs
> 
>

Re: Review Request 61553: RANGER-1730 : Utility script that will list the users with a given role

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61553/#review185372
---




security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 63 (patched)
<https://reviews.apache.org/r/61553/#comment261695>

name variables as userRole. Follow this pattern



security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 133 (patched)
<https://reviews.apache.org/r/61553/#comment261684>

Please Indent the methods to start at same position, check all the methods



security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 141 (patched)
<https://reviews.apache.org/r/61553/#comment261686>

Please use Apache CollectionUtils.isEmpty() if possible. Check all the 
other occurances of other similar checks



security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 206 (patched)
<https://reviews.apache.org/r/61553/#comment261690>

Please check if currentEncryptedPassword can be null?



security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 209 (patched)
<https://reviews.apache.org/r/61553/#comment261691>

can existingRole be null? Please verify



security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 217 (patched)
<https://reviews.apache.org/r/61553/#comment261692>

if UserRole is going to be not null 
do UserRole.equalsIgnoreCase(existingRole.get(0)). Verify similar check and 
correct it where ever needed.



security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 242 (patched)
<https://reviews.apache.org/r/61553/#comment261682>

Change this flag variable to "isUserAuthorized"



security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
Lines 247 (patched)
<https://reviews.apache.org/r/61553/#comment261683>

Authorized?


- Ramesh Mani


On Sept. 14, 2017, 4:33 a.m., Fatima Khan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61553/
> ---
> 
> (Updated Sept. 14, 2017, 4:33 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1730
> https://issues.apache.org/jira/browse/RANGER-1730
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Actual :
> Provide utility to list user according to role.
> 
> Expected :
> Utility to list users for the given role based on thier authorization
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/rolebasedusersearchutil.py PRE-CREATION 
>   
> security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
>  PRE-CREATION 
>   
> security-admin/src/test/java/org/apache/ranger/patch/cliutil/TestRoleBasedUserSearchUtil.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml cb1aad2 
> 
> 
> Diff: https://reviews.apache.org/r/61553/diff/3/
> 
> 
> Testing
> ---
> 
> Tested on Simple against all roles
> Tested on Secure against all roles
> 
> 
> Thanks,
> 
> Fatima Khan
> 
>

Re: Review Request 61553: RANGER-1730 : Utility script that will list the users with a given role

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61553/#review185650
---


Ship it!




Ship It!

- Ramesh Mani


On Sept. 15, 2017, 10:17 a.m., Fatima Khan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61553/
> ---
> 
> (Updated Sept. 15, 2017, 10:17 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1730
> https://issues.apache.org/jira/browse/RANGER-1730
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Actual :
> Provide utility to list user according to role.
> 
> Expected :
> Utility to list users for the given role based on thier authorization
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/rolebasedusersearchutil.py PRE-CREATION 
>   
> security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java
>  PRE-CREATION 
>   
> security-admin/src/test/java/org/apache/ranger/patch/cliutil/TestRoleBasedUserSearchUtil.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml cb1aad2 
> 
> 
> Diff: https://reviews.apache.org/r/61553/diff/4/
> 
> 
> Testing
> ---
> 
> Tested on Simple against all roles
> Tested on Secure against all roles
> 
> 
> Thanks,
> 
> Fatima Khan
> 
>

Re: Review Request 62437: RANGER-1779 : last resource gets duplicated during update policy if policy is created through public api rest call

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62437/#review185812
---




security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
Line 2929 (original), 2930 (patched)
<https://reviews.apache.org/r/62437/#comment262157>

Please conside doing Set uniqueValues = new 
LinkedHashSet<>(values); and remove the code for finding the uniqueValues.
You can trim the output later when you sent it if needed.


- Ramesh Mani


On Sept. 20, 2017, 12:53 p.m., Nikhil P wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62437/
> ---
> 
> (Updated Sept. 20, 2017, 12:53 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1779
> https://issues.apache.org/jira/browse/RANGER-1779
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> 1) create a policy with multiple resource *,default using public api
> 2) go to ranger admin ui and update the policy without any change
> 3) again view the policy.
> Issue:
> default gets duplicated as resource in the policy.
> and even new entry is added in resource map table for the last resource.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
> 63fdf4f 
>   
> security-admin/src/main/java/org/apache/ranger/patch/PatchForNifiResourceUpdateExclude_J10008.java
>  634082c 
> 
> 
> Diff: https://reviews.apache.org/r/62437/diff/1/
> 
> 
> Testing
> ---
> 
> 1)Verified if same resource does not get duplicated during create and update 
> policy.
> 2)Verified if resource duplication does not happen through public API and 
> Public APIv2 as well.
> 3)Verified if policies are getting created with multiple distinct resource.
> 
> 
> Thanks,
> 
> Nikhil P
> 
>

Re: Review Request 62024: RANGER-1756: Handle role related restrictions for users having User role.

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62024/#review186457
---




security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java
Lines 353 (patched)
<https://reviews.apache.org/r/62024/#comment263009>

why you need to create VXUserList() instance, just return the value from  
xUserMgr.searchXUsers(searchCriteria) as it was earlier.



security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java
Line 361 (original), 370 (patched)
<https://reviews.apache.org/r/62024/#comment263010>

will CollectionUtils.size be more 1?, if so change this check 
CollectionUtils.size(userRolesList) == 1  to 
CollectionUtils.size(userRolesList) > 0


- Ramesh Mani


On Sept. 27, 2017, 11:27 a.m., Nitin Galave wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62024/
> ---
> 
> (Updated Sept. 27, 2017, 11:27 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Fatima Khan, Gautam Borad, Mehul 
> Parikh, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1756
> https://issues.apache.org/jira/browse/RANGER-1756
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Handle role related restrictions for users having User role.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/hadoop/security/SecureClientLogin.java 
> 320a9a4 
>   security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 739ea05 
>   security-admin/src/main/webapp/scripts/utils/XAUtils.js ecf43ad 
> 
> 
> Diff: https://reviews.apache.org/r/62024/diff/4/
> 
> 
> Testing
> ---
> 
> Verified scenario's :
> 1. A user with ROLE_ADMIN able to see users which has USER_ROLE/ADMIN_ROLE.
> 2. A user with ROLE_KEYADMIN able to see users which has 
> USER_ROLE/KEYADMIN_ROLE.
> 3. A user with role ROLE_USER able to see only himself.
> 
> 
> Thanks,
> 
> Nitin Galave
> 
>

Re: Review Request 62709: RANGER-1817 : Audit to Solr fails to log when the number of columns are in large number

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62709/#review186786
---




security-admin/contrib/solr_for_audit_setup/conf/managed-schema
Lines 32 (patched)
<https://reviews.apache.org/r/62709/#comment263620>

We need to have a default value for this config in ranger, which can be 
overridden.

Default can be a much less value say 100, which can be rendered on UI 
without any performance issue.


- Ramesh Mani


On Sept. 30, 2017, 6:26 a.m., Fatima Khan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62709/
> ---
> 
> (Updated Sept. 30, 2017, 6:26 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1817
> https://issues.apache.org/jira/browse/RANGER-1817
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Audit to Solr fails to log when the number of columns are in large number. 
> This is due to Solr has a hard limit on solr.StrField and if this string is 
> exceeding max length 32766, it throws exception which causes the audit to 
> fail. To overcome this we need to trip this in Audit records and the best 
> place to do it is in solr schema for ranger-audits.
> 
> For this we need to change the file managed_schema in ranger and commit it to 
> zookeeper.
> Change required in the managed_schema file is, find the following in the 
> managed_schema file and add this param to limit the length to 2500 max.
> 
> 
> Diffs
> -
> 
>   security-admin/contrib/solr_for_audit_setup/conf/managed-schema ee1d894 
> 
> 
> Diff: https://reviews.apache.org/r/62709/diff/1/
> 
> 
> Testing
> ---
> 
> 1.Tested Ranger installation
> 2.Junit test was successful
> 
> 
> Thanks,
> 
> Fatima Khan
> 
>

Re: Review Request 62710: RANGER-1810:Ranger supports plugin to enable, monitor and manage apache Sqoop2

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186865
---




plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
Lines 99 (patched)
<https://reviews.apache.org/r/62710/#comment263728>

can privilege be null? Is it test for this case?
Please add some unit test also for various scenarios.


- Ramesh Mani


On Sept. 30, 2017, 8:34 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> ---
> 
> (Updated Sept. 30, 2017, 8:34 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Apache Sqoop is a tool designed for efficiently transferring bulk data 
> between Apache Hadoop and structured datastores such as relational databases. 
> You can use Sqoop to import data from external structured datastores into 
> Hadoop Distributed File System or related systems like Hive and HBase. 
> Conversely, Sqoop can be used to extract data from Hadoop and export it to 
> external structured datastores such as relational databases and enterprise 
> data warehouses.It successfully graduated from the Incubator in March of 2012 
> and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it 
> supports sqoop authorization. So we should develop sqoop plugin to enable, 
> monitor and manage apache Sqoop2.
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json 
> PRE-CREATION 
>   plugin-sqoop/.gitignore PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION 
>   plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION 
>   plugin-sqoop/pom.xml PRE-CREATION 
>   plugin-sqoop/scripts/install.properties PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
>  PRE-CREATION 
>   
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-sqoop-plugin-shim/.gitignore PRE-CREATION 
>   ranger-sqoop-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-sqoop.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62710/diff/1/
> 
> 
> Testing
> ---
> 
> Our test specialists have rigorously tested this feature.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Re: Review Request 62709: RANGER-1817 : Audit to Solr fails to log when the number of columns are in large number

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62709/#review187051
---


Ship it!




Ship It!

- Ramesh Mani


On Sept. 30, 2017, 6:26 a.m., Fatima Khan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62709/
> ---
> 
> (Updated Sept. 30, 2017, 6:26 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1817
> https://issues.apache.org/jira/browse/RANGER-1817
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Audit to Solr fails to log when the number of columns are in large number. 
> This is due to Solr has a hard limit on solr.StrField and if this string is 
> exceeding max length 32766, it throws exception which causes the audit to 
> fail. To overcome this we need to trip this in Audit records and the best 
> place to do it is in solr schema for ranger-audits.
> 
> For this we need to change the file managed_schema in ranger and commit it to 
> zookeeper.
> Change required in the managed_schema file is, find the following in the 
> managed_schema file and add this param to limit the length to 2500 max.
> 
> 
> Diffs
> -
> 
>   security-admin/contrib/solr_for_audit_setup/conf/managed-schema ee1d894 
> 
> 
> Diff: https://reviews.apache.org/r/62709/diff/1/
> 
> 
> Testing
> ---
> 
> 1.Tested Ranger installation
> 2.Junit test was successful
> 
> 
> Thanks,
> 
> Fatima Khan
> 
>

Re: Review Request 62971: LOG class is imported error for RangerServiceService class

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62971/#review188005
---


Ship it!




Ship It!

- Ramesh Mani


On Oct. 13, 2017, 10 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62971/
> ---
> 
> (Updated Oct. 13, 2017, 10 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1836
> https://issues.apache.org/jira/browse/RANGER-1836
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> LOG class is imported error for RangerServiceService class
> RangerServiceService.java import:
> import java.util.logging.Logger;
> I think java.util.logging.Logger class should be repalced with:
> import org.apache.commons.logging.Log;
> import org.apache.commons.logging.LogFactory;
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerServiceService.java
>  3dd761a2 
> 
> 
> Diff: https://reviews.apache.org/r/62971/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Re: Review Request 63055: Audit log records for 'use dbName' and 'show databases' hive commands contain large number of tags

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63055/#review188238
---


Ship it!




Ship It!

- Ramesh Mani


On Oct. 16, 2017, 11:20 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63055/
> ---
> 
> (Updated Oct. 16, 2017, 11:20 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-1841
> https://issues.apache.org/jira/browse/RANGER-1841
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> When a Hive service is configured for tag-based authorization, the audit log 
> generated for ‘use dbName’ or 'show databases' command would contain all the 
> tags associated with: the database, all tables in the database, all the 
> columns in the database. The number of tags in this audit log could be too 
> many; and having such large number of tags in audit logs of 'use ' 
> command may not be useful. It will be better not to log tags in audit logs 
> for 'use ' commands. Policy-id recorded in the audit log can be used 
> to identity the tag, if a tag-based policy authorized the command.
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuditHandler.java
>  9dea37a 
> 
> 
> Diff: https://reviews.apache.org/r/63055/diff/1/
> 
> 
> Testing
> ---
> 
> Tested with local VM
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/
---

Review request for ranger, Don Bosco Durai and Madhan Neethiraj.


Repository: ranger


Description
---

RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format


Diffs
-

  agents-audit/README.txt PRE-CREATION 
  agents-audit/pom.xml c8bd1d8 
  
agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
 66d8504 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
 314b130 
  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
eff3824 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/AuditWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/TextWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
PRE-CREATION 
  pom.xml 589cd6a 
  src/main/assembly/hdfs-agent.xml 5279a9a 


Diff: https://reviews.apache.org/r/63552/diff/1/


Testing
---

Testing done in local


Thanks,

Ramesh Mani

Re: Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/
---

(Updated Nov. 4, 2017, 12:36 a.m.)


Review request for ranger, Don Bosco Durai and Madhan Neethiraj.


Changes
---

added all plugin to pack hive-exec jar as dependency for ranger audit to hdfs 
orc file creation


Repository: ranger


Description
---

RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format


Diffs (updated)
-

  agents-audit/README.txt PRE-CREATION 
  agents-audit/pom.xml c8bd1d8 
  
agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
 66d8504 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
 314b130 
  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
eff3824 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/AuditWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/TextWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
PRE-CREATION 
  pom.xml 589cd6a 
  src/main/assembly/hdfs-agent.xml 5279a9a 


Diff: https://reviews.apache.org/r/63552/diff/2/

Changes: https://reviews.apache.org/r/63552/diff/1-2/


Testing
---

Testing done in local


Thanks,

Ramesh Mani

Re: Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/
---

(Updated Nov. 4, 2017, 12:38 a.m.)


Review request for ranger, Don Bosco Durai and Madhan Neethiraj.


Repository: ranger


Description
---

RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format


Diffs (updated)
-

  agents-audit/README.txt PRE-CREATION 
  agents-audit/pom.xml c8bd1d8 
  
agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
 66d8504 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
 314b130 
  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
eff3824 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/AuditWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/TextWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
PRE-CREATION 
  pom.xml 589cd6a 
  src/main/assembly/hbase-agent.xml 3ebc334 
  src/main/assembly/hdfs-agent.xml 5279a9a 
  src/main/assembly/hive-agent.xml ca65c80 
  src/main/assembly/knox-agent.xml 8357d49 
  src/main/assembly/plugin-atlas.xml fd98811 
  src/main/assembly/plugin-kafka.xml 95855d9 
  src/main/assembly/plugin-kms.xml 6d15f2a 
  src/main/assembly/plugin-solr.xml de30bfb 
  src/main/assembly/plugin-sqoop.xml d2bd69a 
  src/main/assembly/plugin-yarn.xml c6a48e8 
  src/main/assembly/storm-agent.xml 64224ec 


Diff: https://reviews.apache.org/r/63552/diff/3/

Changes: https://reviews.apache.org/r/63552/diff/2-3/


Testing
---

Testing done in local


Thanks,

Ramesh Mani

Re: Plugin Shim question

[Ramesh Mani]

Colm,

Yes look like we need to have multiple shims.

To support multiple version of Hadoop, we may need a super shim which will
find which version of Hadoop ( based on method signature) is used and then
instantiate the corresponding  sub shim RangerYarnAuthorizer.

This may be the case for HDFS plugin also in there is a different
behaviors or signature change in authorization hook.

Thanks,
Ramesh


On 11/9/17, 4:22 AM, "Colm O hEigeartaigh"  wrote:

>Hi all,
>
>I'm working on adding support for Hadoop 3.0.0 to the Yarn component
>(RANGER-1738).
>
>YarnAuthorizationProvider has some updated methods in Hadoop 3.0.0. It's
>easy to work around this in the RangerYarnAuthorizer though, so that it
>supports both Hadoop 2.7.x, 2.8.x and 3.0.0.
>
>The problem is in the plugin shim code for RangerYarnAuthorizer. It
>delegates the calls to the underlying YarnAuthorizationProvider instance.
>This means it's not possible to support both 2.7.x and 3.0.0 as it is in
>the plugin version of RangerYarnAuthorizer.
>
>Any ideas on this? The only way I can think of supporting it is to have
>separate plugin shims for Hadoop 2 + 3.
>
>Colm.
>
>
>-- 
>Colm O hEigeartaigh
>
>Talend Community Coder
>http://coders.talend.com

Re: Review Request 63785: RANGER-1884 : Default Policy is not created for Ranger KMS and Tag service

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63785/#review190978
---


Ship it!




Ship It!

- Ramesh Mani


On Nov. 14, 2017, 4:59 a.m., bhavik patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63785/
> ---
> 
> (Updated Nov. 14, 2017, 4:59 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan 
> Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1884
> https://issues.apache.org/jira/browse/RANGER-1884
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Default Policy is not created for Ranger KMS as well as for Tag services.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBaseService.java
>  25f9985 
>   
> plugin-kms/src/main/java/org/apache/ranger/services/kms/RangerServiceKMS.java 
> cd368e4 
> 
> 
> Diff: https://reviews.apache.org/r/63785/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Verified default policy is getting created for Ranger Kms & for Tag 
> service.
> 
> 
> Thanks,
> 
> bhavik patel
> 
>

Re: Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/
---

(Updated Nov. 17, 2017, 7:38 p.m.)


Review request for ranger, Don Bosco Durai and Madhan Neethiraj.


Changes
---

Fixed review comment


Repository: ranger


Description
---

RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format


Diffs (updated)
-

  agents-audit/README.txt PRE-CREATION 
  agents-audit/pom.xml c8bd1d8 
  
agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
 66d8504 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
 314b130 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditWriterFactory.java
 PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
eff3824 
  
agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
 41513ba 
  
agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractAuditWriter.java
 PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/JSONWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
PRE-CREATION 
  agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
PRE-CREATION 
  pom.xml 589cd6a 
  src/main/assembly/hbase-agent.xml 3ebc334 
  src/main/assembly/hdfs-agent.xml 5279a9a 
  src/main/assembly/hive-agent.xml ca65c80 
  src/main/assembly/knox-agent.xml 8357d49 
  src/main/assembly/plugin-atlas.xml fd98811 
  src/main/assembly/plugin-kafka.xml 95855d9 
  src/main/assembly/plugin-kms.xml 6d15f2a 
  src/main/assembly/plugin-solr.xml de30bfb 
  src/main/assembly/plugin-sqoop.xml d2bd69a 
  src/main/assembly/plugin-yarn.xml c6a48e8 
  src/main/assembly/storm-agent.xml 64224ec 


Diff: https://reviews.apache.org/r/63552/diff/4/

Changes: https://reviews.apache.org/r/63552/diff/3-4/


Testing
---

Testing done in local


Thanks,

Ramesh Mani

Re: Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

> On Nov. 4, 2017, 2:21 a.m., Abhay Kulkarni wrote:
> > agents-audit/README.txt
> > Lines 44 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880895#file1880895line44>
> >
> > Please clarify if this query needs to be run as a part of Audit 
> > framework set-up.

No, this is going to be manual activity to create a tables. We don't have the 
hive client packed in each of the plugins to do it. I feel its should be fine 
to do it as external script based on  the users necessity


> On Nov. 4, 2017, 2:21 a.m., Abhay Kulkarni wrote:
> > agents-audit/README.txt
> > Lines 68 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880895#file1880895line68>
> >
> > Is the location hard-coded to /ranger/audit/hdfs? or is it configured 
> > through some other configuration variable?

This part of the Audit framework config.


> On Nov. 4, 2017, 2:21 a.m., Abhay Kulkarni wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java
> > Lines 108 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880902#file1880902line108>
> >
> > Clarification needed: Does this method log all events passed to it as 
> > one ORC batch? If so, then if the number of events passed is much smaller 
> > than the optimal batch-size, there may be smaller batches created.

Yes, this is possible. We have three buffer params which I shall discuss in 
detail in JIRA and how to handle this.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/#review190084
---


On Nov. 17, 2017, 7:38 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63552/
> ---
> 
> (Updated Nov. 17, 2017, 7:38 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format
> 
> 
> Diffs
> -
> 
>   agents-audit/README.txt PRE-CREATION 
>   agents-audit/pom.xml c8bd1d8 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
>  66d8504 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
>  314b130 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditWriterFactory.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
> eff3824 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
>  41513ba 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractAuditWriter.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/JSONWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
> PRE-CREATION 
>   pom.xml 589cd6a 
>   src/main/assembly/hbase-agent.xml 3ebc334 
>   src/main/assembly/hdfs-agent.xml 5279a9a 
>   src/main/assembly/hive-agent.xml ca65c80 
>   src/main/assembly/knox-agent.xml 8357d49 
>   src/main/assembly/plugin-atlas.xml fd98811 
>   src/main/assembly/plugin-kafka.xml 95855d9 
>   src/main/assembly/plugin-kms.xml 6d15f2a 
>   src/main/assembly/plugin-solr.xml de30bfb 
>   src/main/assembly/plugin-sqoop.xml d2bd69a 
>   src/main/assembly/plugin-yarn.xml c6a48e8 
>   src/main/assembly/storm-agent.xml 64224ec 
> 
> 
> Diff: https://reviews.apache.org/r/63552/diff/4/
> 
> 
> Testing
> ---
> 
> Testing done in local
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

> On Nov. 11, 2017, 6:15 p.m., Kevin Risden wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
> > Line 62 (original), 62 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880898#file1880898line62>
> >
> > Is this a bug regardless of the ORC change?

Yes this is a bug, it just logs a wrong message. so fixed it


> On Nov. 11, 2017, 6:15 p.m., Kevin Risden wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/utils/AuditWriter.java
> > Lines 145 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880900#file1880900line145>
> >
> > Will these folders have the correct permissions?

yes, this is the existing functionality for creating the folders for audit logs.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/#review190782
-------


On Nov. 17, 2017, 7:38 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63552/
> ---
> 
> (Updated Nov. 17, 2017, 7:38 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format
> 
> 
> Diffs
> -
> 
>   agents-audit/README.txt PRE-CREATION 
>   agents-audit/pom.xml c8bd1d8 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
>  66d8504 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
>  314b130 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditWriterFactory.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
> eff3824 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
>  41513ba 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractAuditWriter.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/JSONWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
> PRE-CREATION 
>   pom.xml 589cd6a 
>   src/main/assembly/hbase-agent.xml 3ebc334 
>   src/main/assembly/hdfs-agent.xml 5279a9a 
>   src/main/assembly/hive-agent.xml ca65c80 
>   src/main/assembly/knox-agent.xml 8357d49 
>   src/main/assembly/plugin-atlas.xml fd98811 
>   src/main/assembly/plugin-kafka.xml 95855d9 
>   src/main/assembly/plugin-kms.xml 6d15f2a 
>   src/main/assembly/plugin-solr.xml de30bfb 
>   src/main/assembly/plugin-sqoop.xml d2bd69a 
>   src/main/assembly/plugin-yarn.xml c6a48e8 
>   src/main/assembly/storm-agent.xml 64224ec 
> 
> 
> Diff: https://reviews.apache.org/r/63552/diff/4/
> 
> 
> Testing
> ---
> 
> Testing done in local
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

> On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote:
> > agents-audit/README.txt
> > Lines 28 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880895#file1880895line28>
> >
> > I assume, which this option, memory buffer won't be used.

Bosco, memory buffer is not removed, AuditFileCached provider is going to fetch 
data from the file in buffer sizes and send it AuditQueue only in the current 
pipeline of data flow. I shall details this in JIRA and we can review this as 
this is significant in what is support and what not.


> On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/utils/AuditWriter.java
> > Lines 97 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880900#file1880900line97>
> >
> > This class seems more HDFS specific. Should we just call it 
> > HDFSAuditWriter?
> > 
> > I think, we have two variables:
> > 1. FileSystem (Local File, HDFS, S3, etc)
> > 2. Format/encoding (JSON, ORC, Parquet, Avro, etc.)
> > 
> > We should be able to mix and match where possible.

FileSystem API of Hadoop core is handling all the fileSystem, so it just the 
param value what is configured in the AuditFramework will determine the 
destination like HDFS,LocalFile, S3,etc) and Ranger AuditWriter will handle 
fileformat of the audit.


> On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java
> > Lines 154 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880901#file1880901line154>
> >
> > Should we log and ignore this Exception

This is done for purpose, so that the calling source can do the deffered count 
and log it.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/#review190104
---


On Nov. 17, 2017, 7:38 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63552/
> ---
> 
> (Updated Nov. 17, 2017, 7:38 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format
> 
> 
> Diffs
> -
> 
>   agents-audit/README.txt PRE-CREATION 
>   agents-audit/pom.xml c8bd1d8 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
>  66d8504 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
>  314b130 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditWriterFactory.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
> eff3824 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
>  41513ba 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractAuditWriter.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/JSONWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
> PRE-CREATION 
>   pom.xml 589cd6a 
>   src/main/assembly/hbase-agent.xml 3ebc334 
>   src/main/assembly/hdfs-agent.xml 5279a9a 
>   src/main/assembly/hive-agent.xml ca65c80 
>   src/main/assembly/knox-agent.xml 8357d49 
>   src/main/assembly/plugin-atlas.xml fd98811 
>   src/main/assembly/plugin-kafka.xml 95855d9 
>   src/main/assembly/plugin-kms.xml 6d15f2a 
>   src/main/assembly/plugin-solr.xml de30bfb 
>   src/main/assembly/plugin-sqoop.xml d2bd69a 
>   src/main/assembly/plugin-yarn.xml c6a48e8 
>   src/main/assembly/storm-agent.xml 64224ec 
> 
> 
> Diff: https://reviews.apache.org/r/63552/diff/4/
> 
> 
> Testing
> ---
> 
> Testing done in local
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 63552: RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format

[Ramesh Mani]

> On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote:
> > agents-audit/README.txt
> > Lines 40 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880895#file1880895line40>
> >
> > We need to check with the ORC team to see what would be the ideal (raw) 
> > file size.

pinged ORC team in the JIRA


> On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
> > Lines 33 (patched)
> > <https://reviews.apache.org/r/63552/diff/3/?file=1880897#file1880897line39>
> >
> > Should I assume that this is JSONWriter?

Renamed this Class to JSONWriter


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63552/#review190104
-------


On Nov. 17, 2017, 7:38 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63552/
> ---
> 
> (Updated Nov. 17, 2017, 7:38 p.m.)
> 
> 
> Review request for ranger, Don Bosco Durai and Madhan Neethiraj.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format
> 
> 
> Diffs
> -
> 
>   agents-audit/README.txt PRE-CREATION 
>   agents-audit/pom.xml c8bd1d8 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
>  66d8504 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java
>  314b130 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditWriterFactory.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 
> eff3824 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java
>  41513ba 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractAuditWriter.java
>  PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/JSONWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java 
> PRE-CREATION 
>   agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java 
> PRE-CREATION 
>   pom.xml 589cd6a 
>   src/main/assembly/hbase-agent.xml 3ebc334 
>   src/main/assembly/hdfs-agent.xml 5279a9a 
>   src/main/assembly/hive-agent.xml ca65c80 
>   src/main/assembly/knox-agent.xml 8357d49 
>   src/main/assembly/plugin-atlas.xml fd98811 
>   src/main/assembly/plugin-kafka.xml 95855d9 
>   src/main/assembly/plugin-kms.xml 6d15f2a 
>   src/main/assembly/plugin-solr.xml de30bfb 
>   src/main/assembly/plugin-sqoop.xml d2bd69a 
>   src/main/assembly/plugin-yarn.xml c6a48e8 
>   src/main/assembly/storm-agent.xml 64224ec 
> 
> 
> Diff: https://reviews.apache.org/r/63552/diff/4/
> 
> 
> Testing
> ---
> 
> Testing done in local
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 63919: RANGER-1895 - Simplify Storm dependencies

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63919/#review191915
---




storm-agent/pom.xml
Lines 95-96 (original)
<https://reviews.apache.org/r/63919/#comment269849>

Colm, did you verify that audit to solr is working as expected? We may need 
this solr jar for Ranger audit dependency


- Ramesh Mani


On Nov. 27, 2017, 3:27 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63919/
> ---
> 
> (Updated Nov. 27, 2017, 3:27 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1895
> https://issues.apache.org/jira/browse/RANGER-1895
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> This task is to simplify the Apache Storm dependencies for Ranger. We are 
> shipping the hadoop-hdfs jar, which is not required. Secondly, we can avoid 
> explicitly listing some of the dependencies, as they get pulled in via other 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ranger-storm-plugin-shim/pom.xml de1972d4 
>   storm-agent/pom.xml 6e74e5b3 
> 
> 
> Diff: https://reviews.apache.org/r/63919/diff/3/
> 
> 
> Testing
> ---
> 
> Tested the plugin works OK with Apache Storm 1.1.1. The only change in the 
> distribution is that it doesn't include the Hadoop HDFS jar.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>