[jira] [Commented] (RANGER-1938) Solr for Audit setup doesn't use DocValues effectively

2017-12-20 Thread Don Bosco Durai (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16299544#comment-16299544
 ] 

Don Bosco Durai commented on RANGER-1938:
-

bq. Does it make sense for me to open a JIRA against the Ambari project as well 
to fix this schema? I can link to this JIRA since they are related.
Yes it would be good to create the JIRA. The Ranger members in Ambari team 
would be able to replicate your changes there.

bq. I don't think there is a great reason to not enable at the "global" 
fieldType level. We can disable at the individual field level if necessary. 
We can do either way. We can explicitly set the docValues for each field or do 
the global setting. If we are going to do global changes, then let's document 
it in the file itself, so anyone looking into it in the future will be aware of 
the reason.

Thanks for the details of your environment. It is very useful.


> Solr for Audit setup doesn't use DocValues effectively
> --
>
> Key: RANGER-1938
> URL: https://issues.apache.org/jira/browse/RANGER-1938
> Project: Ranger
>  Issue Type: Improvement
>  Components: audit
>Affects Versions: 0.6.0, 0.7.0, 0.6.1, 0.6.2, 0.6.3, 0.7.1
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>  Labels: performance
> Fix For: 1.0.0, 0.7.2
>
> Attachments: 
> 0001-RANGER-1938-Enable-DocValues-for-more-fields-in-Solr.patch
>
>
> Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing 
> Ranger Audit events for displaying in Ranger Admin. In our case, we have 
> noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a 
> few other people who are having very similar problems with OOM errors.
> I've typed up some details about how the way Ranger is using Solr requires a 
> lot of heap. I've also outlined the fix for this which significantly reduced 
> the amount of heap memory required. I'm an Apache Lucene/Solr committer so 
> this optimization/usage might not be immediately obvious to those using Solr 
> especially version 5.x.
> https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 64764: RANGER-1941:Use already defined methods and optimized log printing in RangerScriptExecutionContext class

2017-12-20 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64764/
---

Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
Neethiraj, Velmurugan Periasamy, and Qiang Zhang.


Bugs: RANGER-1941
https://issues.apache.org/jira/browse/RANGER-1941


Repository: ranger


Description
---

We have defined the method of log printing in the RangerScriptExecutionContext 
class. Why not use these methods in this class, use the defined method logDebug 
logError instead of LOG.debug and LOG.error to keep the code style and optimize 
the log printing.


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerScriptExecutionContext.java
 0c8df41 


Diff: https://reviews.apache.org/r/64764/diff/1/


Testing
---

Tested it.


Thanks,

pengjianhua

[jira] [Created] (RANGER-1941) Use already defined methods and optimized log printing in RangerScriptExecutionContext class

2017-12-20 Thread peng.jianhua (JIRA) 
peng.jianhua created RANGER-1941:


 Summary: Use already defined methods and optimized log printing in 
RangerScriptExecutionContext class
 Key: RANGER-1941
 URL: https://issues.apache.org/jira/browse/RANGER-1941
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Affects Versions: master
Reporter: peng.jianhua
Assignee: peng.jianhua
Priority: Minor
 Fix For: master


We have defined the method of log printing in the RangerScriptExecutionContext 
class. Why not use these methods in this class, use the defined method logDebug 
logError instead of LOG.debug and LOG.error to keep the code style and optimize 
the log printing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (RANGER-1929) The ranger should support the View policy.

2017-12-20 Thread Qiang Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang resolved RANGER-1929.
-
   Resolution: Fixed
Fix Version/s: master
   1.0.0

> The ranger should support the View policy.
> --
>
> Key: RANGER-1929
> URL: https://issues.apache.org/jira/browse/RANGER-1929
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>  Labels: newbie, patch
> Fix For: 1.0.0, master
>
> Attachments: 
> 0001-RANGER-1929-The-ranger-should-support-the-View-polic.patch, 
> ragner_pic1.png, ragner_pic2.png, ragner_pic3.png, ragner_pic4.png, 
> view-policy.png
>
>
> Currently we can only edit the policy without viewing the policy. We must use 
> editing funtion of policy when only need to query the detail for policy. So 
> we should supply the function of the query detail for policy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64723: Ranger tagsync should process ENTITY_CREATE notification, to support Atlas import feature

2017-12-20 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64723/#review194289
---


Ship it!




Ship It!

- pengjianhua


On 十二月 19, 2017, 9:07 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64723/
> ---
> 
> (Updated 十二月 19, 2017, 9:07 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-1937
> https://issues.apache.org/jira/browse/RANGER-1937
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently Ranger tagsync ignores ENTITY_CREATE notifications from Atlas, as 
> no tags would be associated with the newly created entity. However, when an 
> entity is created in Atlas via import, tags could be associated with the 
> entity. To not miss the tags associated with the entities, Ranger tagsync 
> should be updated to process ENTITY_CREATE notifications as well.
> 
> 
> Diffs
> -
> 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
>  f42c908 
> 
> 
> Diff: https://reviews.apache.org/r/64723/diff/1/
> 
> 
> Testing
> ---
> 
> Unit test run successful
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 64723: Ranger tagsync should process ENTITY_CREATE notification, to support Atlas import feature

2017-12-20 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64723/#review194286
---


Ship it!




Ship It!

- Madhan Neethiraj


On Dec. 19, 2017, 9:07 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64723/
> ---
> 
> (Updated Dec. 19, 2017, 9:07 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-1937
> https://issues.apache.org/jira/browse/RANGER-1937
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently Ranger tagsync ignores ENTITY_CREATE notifications from Atlas, as 
> no tags would be associated with the newly created entity. However, when an 
> entity is created in Atlas via import, tags could be associated with the 
> entity. To not miss the tags associated with the entities, Ranger tagsync 
> should be updated to process ENTITY_CREATE notifications as well.
> 
> 
> Diffs
> -
> 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
>  f42c908 
> 
> 
> Diff: https://reviews.apache.org/r/64723/diff/1/
> 
> 
> Testing
> ---
> 
> Unit test run successful
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 64753: Simplify Maven dependencies and assembly specification for hdfs plugin module

2017-12-20 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64753/#review194282
---




hdfs-agent/pom.xml
Line 36 (original), 37 (patched)


Given ranger-hdfs-plugin doesn't depend on its shim, it will be cleaner to 
not have this dependency here.



hdfs-agent/pom.xml
Lines 116 (patched)


hadoop libraries (like hadoop-common, hadoop-auth,  should already be 
present in NameNode classpath. We shouldn't have to package them in 
ranger-hdfs-plugin. Please review.



src/main/assembly/hdfs-agent.xml
Line 47 (original)


These files are needed in install/lib directory for the manuall 
installation to work. If this library list changes, please ensure that manuall 
installation works correctly.


- Madhan Neethiraj


On Dec. 20, 2017, 7:04 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64753/
> ---
> 
> (Updated Dec. 20, 2017, 7:04 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Selvamohan Neethiraj.
> 
> 
> Bugs: RANGER-1939
> https://issues.apache.org/jira/browse/RANGER-1939
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> There are two issues with the Maven POM files for Ranger's hdfs plugin module.
> 1. There are overlapping and sometimes conflicting versions of libraries on 
> which hdfs plugin code depends. Conflicts arise partly because some of the 
> libraries packaged with hdfs plugin module are already exist in hdfs 
> component and have different versions.
> 2. assembly specification for hdfs plugin module uses DependencySets - a 
> construct which is confusing and hard to get right. They also clutter up 
> build output log with spurious messages. It is desirable to use FileSets 
> which are easier to understand and straightforward to specify in an assembly 
> spec.
> 
> 
> Diffs
> -
> 
>   hdfs-agent/pom.xml e9b2b59 
>   pom.xml 255b02a 
>   src/main/assembly/hdfs-agent.xml 561d137 
> 
> 
> Diff: https://reviews.apache.org/r/64753/diff/1/
> 
> 
> Testing
> ---
> 
> Tested with local VM
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 64677: RANGER-1934:Optimize the init method in BaseAuditHandler class to avoid ArrayIndexOutOfBoundsException

2017-12-20 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64677/#review194284
---


Ship it!




Ship It!

- Qiang Zhang


On Dec. 20, 2017, 5:54 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64677/
> ---
> 
> (Updated Dec. 20, 2017, 5:54 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1934
> https://issues.apache.org/jira/browse/RANGER-1934
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Optimize the init method in BaseAuditHandler class to avoid 
> ArrayIndexOutOfBoundsException
> The follow in the init method "   List tokens = 
> MiscUtil.toArray(propPrefix, ".");
> String finalToken = tokens.get(tokens.size() - 1);".
> in the init method we should add " if (tokens.size() > 1)" to avoid 
> ArrayIndexOutOfBoundsException.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/BaseAuditHandler.java
>  b095000 
> 
> 
> Diff: https://reviews.apache.org/r/64677/diff/3/
> 
> 
> Testing
> ---
> 
> Tested it.
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 64677: RANGER-1934:Optimize the init method in BaseAuditHandler class to avoid ArrayIndexOutOfBoundsException

2017-12-20 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64677/#review194272
---


Ship it!




Ship It!

- Ramesh Mani


On Dec. 20, 2017, 5:54 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64677/
> ---
> 
> (Updated Dec. 20, 2017, 5:54 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1934
> https://issues.apache.org/jira/browse/RANGER-1934
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Optimize the init method in BaseAuditHandler class to avoid 
> ArrayIndexOutOfBoundsException
> The follow in the init method "   List tokens = 
> MiscUtil.toArray(propPrefix, ".");
> String finalToken = tokens.get(tokens.size() - 1);".
> in the init method we should add " if (tokens.size() > 1)" to avoid 
> ArrayIndexOutOfBoundsException.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/BaseAuditHandler.java
>  b095000 
> 
> 
> Diff: https://reviews.apache.org/r/64677/diff/3/
> 
> 
> Testing
> ---
> 
> Tested it.
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Review Request 64753: Simplify Maven dependencies and assembly specification for hdfs plugin module

2017-12-20 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64753/
---

Review request for ranger, Madhan Neethiraj and Selvamohan Neethiraj.


Bugs: RANGER-1939
https://issues.apache.org/jira/browse/RANGER-1939


Repository: ranger


Description
---

There are two issues with the Maven POM files for Ranger's hdfs plugin module.
1. There are overlapping and sometimes conflicting versions of libraries on 
which hdfs plugin code depends. Conflicts arise partly because some of the 
libraries packaged with hdfs plugin module are already exist in hdfs component 
and have different versions.
2. assembly specification for hdfs plugin module uses DependencySets - a 
construct which is confusing and hard to get right. They also clutter up build 
output log with spurious messages. It is desirable to use FileSets which are 
easier to understand and straightforward to specify in an assembly spec.


Diffs
-

  hdfs-agent/pom.xml e9b2b59 
  pom.xml 255b02a 
  src/main/assembly/hdfs-agent.xml 561d137 


Diff: https://reviews.apache.org/r/64753/diff/1/


Testing
---

Tested with local VM


Thanks,

Abhay Kulkarni

[jira] [Commented] (RANGER-1938) Solr for Audit setup doesn't use DocValues effectively

2017-12-20 Thread Kevin Risden (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298628#comment-16298628
 ] 

Kevin Risden commented on RANGER-1938:
--

bq. If Apache Ambari is used to manage Ranger, then Ambari has its own template 
for solr-config

I think this only takes affect during initial install if using Solr cloud. The 
template isn't pushed back up to Zookeeper. I know my changes haven't been 
overwritten by Ambari (which I was a bit surprised by).

Does it make sense for me to open a JIRA against the Ambari project as well to 
fix this schema? I can link to this JIRA since they are related.

bq. If you are using Ambari-Infra Solr, then it is shared by Ambari LogSearch 
and Apache Atlas

It is true that Ambari Infra Solr is shared by Ranger, LogSearch, and Atlas. 
The modified Solr config is only for the single collection which is 
ranger_audit in this case. This could also be a problem with LogSearch and 
Atlas and would have to be fixed separately. We don't have LogSearch enabled 
right now and Atlas has too small of a dataset for us right now to notice a 
problem.

bq. We have to emphasize that changing the schema requires rebuilding of Solr 
Collection. Which means all existing audits from Solr will be deleted. There is 
an option to rebuild it from the audits stored in HDFS, but currently, there 
are no known documentation or scripts for that

Since the TTL is small, deleting is what I recommend since it avoids the 
complexity of reindexing from HDFS. If necessary definitely could reindex from 
HDFS but as you said this would require more effort.
 
bq. I have given review comments at review board. Essentially, I would prefer 
to set the docValues at individual field level, rather than at global/default 
fieldType level.

I don't think there is a great reason to not enable at the "global" fieldType 
level. We can disable at the individual field level if necessary. There are 
very few cases where turning off DocValues makes sense especially for Ranger. 
DocValues will take up more disk space during indexing but the tradeoff is that 
Solr is stable during any sorting or faceting operation. Most of the queries 
I've seen for Ranger use sorting in the Ranger Admin UI. The thing to remember 
here is that DocValues is at the collection level global and not for Solr 
global. Each collection can have a different configuration.

bq. I have a request. I know it is very difficult to recommend configuration 
settings for Solr. With your current setup, can you share the configuration you 
currently have?

1. Memory setting for Solr = *4GB (after change for this JIRA. Previously had 
up to 24GB and Solr wasn't stable over time.)*
2. Number of shards and replication = *5 shards - no replication right now 
(single Solr node)*
3. Number of days for TTL = *21 days*
4. Max number of documents (based on TTL) = *~1.33 billion live docs and ~110 
million deleted docs for 21 days. Split over 5 shards - each shard ~266 million 
live docs and ~22 million deleted docs right now.*
5. Are Solr instances running on dedicated servers or one of the master 
servers? = *1 Solr instance running on a data node with spinning disk (not 
using HDFS right now)*

Some other stats that could be helpful:
* very few queries against Ambari Infra Solr - Ranger Admin UI and Atlas UI 
(very rarely)
* default GC tuning from Solr scripts are used
* current uptime 2 weeks - no full GC pauses with 4GB heap
* average pause time is .00118s
* average pause interval is 15.41s
* average heap usage 2GB


> Solr for Audit setup doesn't use DocValues effectively
> --
>
> Key: RANGER-1938
> URL: https://issues.apache.org/jira/browse/RANGER-1938
> Project: Ranger
>  Issue Type: Bug
>  Components: audit
>Affects Versions: 0.6.0, 0.7.0, 0.6.1, 0.6.2, 0.6.3, 0.7.1
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>  Labels: performance
> Fix For: 1.0.0, 0.7.2
>
> Attachments: 
> 0001-RANGER-1938-Enable-DocValues-for-more-fields-in-Solr.patch
>
>
> Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing 
> Ranger Audit events for displaying in Ranger Admin. In our case, we have 
> noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a 
> few other people who are having very similar problems with OOM errors.
> I've typed up some details about how the way Ranger is using Solr requires a 
> lot of heap. I've also outlined the fix for this which significantly reduced 
> the amount of heap memory required. I'm an Apache Lucene/Solr committer so 
> this optimization/usage might not be immediately obvious to those using Solr 
> especially version 5.x.
> https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html



--
This message was sent by Atlassian JIRA

[jira] [Commented] (RANGER-1938) Solr for Audit setup doesn't use DocValues effectively

2017-12-20 Thread Kevin Risden (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298612#comment-16298612
 ] 

Kevin Risden commented on RANGER-1938:
--

[~bosco] - Thanks for the feedback. I don't see your comments on review board? 
I'll get the answers to your questions as well.

> Solr for Audit setup doesn't use DocValues effectively
> --
>
> Key: RANGER-1938
> URL: https://issues.apache.org/jira/browse/RANGER-1938
> Project: Ranger
>  Issue Type: Bug
>  Components: audit
>Affects Versions: 0.6.0, 0.7.0, 0.6.1, 0.6.2, 0.6.3, 0.7.1
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>  Labels: performance
> Fix For: 1.0.0, 0.7.2
>
> Attachments: 
> 0001-RANGER-1938-Enable-DocValues-for-more-fields-in-Solr.patch
>
>
> Ranger uses Ambari Infra Solr (or another Apache Solr install) for storing 
> Ranger Audit events for displaying in Ranger Admin. In our case, we have 
> noticed quite a few Ambari Infra Solr OOM due to Ranger. I've talked with a 
> few other people who are having very similar problems with OOM errors.
> I've typed up some details about how the way Ranger is using Solr requires a 
> lot of heap. I've also outlined the fix for this which significantly reduced 
> the amount of heap memory required. I'm an Apache Lucene/Solr committer so 
> this optimization/usage might not be immediately obvious to those using Solr 
> especially version 5.x.
> https://risdenk.github.io/2017/12/18/ambari-infra-solr-ranger.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1781) RangerUI :Policy create/edit form should display only relevant accesses based on the user-selected resource.

2017-12-20 Thread Nitin Galave (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298416#comment-16298416
 ] 

Nitin Galave commented on RANGER-1781:
--

UI patch is committed to apache [master| 
https://github.com/apache/ranger/commit/f1fb6315f0d4cbc1c32b93d7951fb99ea376c73a]
 branch.

> RangerUI :Policy create/edit form should display only relevant accesses based 
> on the user-selected resource.
> 
>
> Key: RANGER-1781
> URL: https://issues.apache.org/jira/browse/RANGER-1781
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 1.0.0
>Reporter: Nitin Galave
>Assignee: Nitin Galave
> Fix For: 1.0.0
>
> Attachments: RANGER-1781.patch
>
>
> Policy create/edit form should display only applicable set of access 
> permissions based on the policy resource (excludedAccesses property) and not 
> the entire set of permissions defined for the service definition.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (RANGER-1940) Upgrade to Knox 0.14.0

2017-12-20 Thread Colm O hEigeartaigh (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colm O hEigeartaigh updated RANGER-1940:

Attachment: 0001-RANGER-1940-Upgrade-to-Knox-0.14.0.patch

> Upgrade to Knox 0.14.0
> --
>
> Key: RANGER-1940
> URL: https://issues.apache.org/jira/browse/RANGER-1940
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
> Fix For: 1.0.0
>
> Attachments: 0001-RANGER-1940-Upgrade-to-Knox-0.14.0.patch
>
>
> This task is to upgrade to Knox 0.14.0. We can take advantage of the changes 
> made to the GatewayTestDriver to simplify the test configuration as a result.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 64746: RANGER-1940 - Upgrade to Knox 0.14.0

2017-12-20 Thread Colm O hEigeartaigh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64746/
---

Review request for ranger.


Bugs: RANGER-1940
https://issues.apache.org/jira/browse/RANGER-1940


Repository: ranger


Description
---

This task is to upgrade to Knox 0.14.0. We can take advantage of the changes 
made to the GatewayTestDriver to simplify the test configuration as a result.


Diffs
-

  knox-agent/src/test/java/org/apache/ranger/services/knox/KnoxRangerTest.java 
53e66df4 
  pom.xml 255b02aa 


Diff: https://reviews.apache.org/r/64746/diff/1/


Testing
---


Thanks,

Colm O hEigeartaigh

[jira] [Created] (RANGER-1940) Upgrade to Knox 0.14.0

2017-12-20 Thread Colm O hEigeartaigh (JIRA) 
Colm O hEigeartaigh created RANGER-1940:
---

 Summary: Upgrade to Knox 0.14.0
 Key: RANGER-1940
 URL: https://issues.apache.org/jira/browse/RANGER-1940
 Project: Ranger
  Issue Type: Improvement
  Components: plugins
Reporter: Colm O hEigeartaigh
Assignee: Colm O hEigeartaigh
 Fix For: 1.0.0


This task is to upgrade to Knox 0.14.0. We can take advantage of the changes 
made to the GatewayTestDriver to simplify the test configuration as a result.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64634: RANGER-1929:The ranger should support the View policy.

2017-12-20 Thread Colm O hEigeartaigh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64634/#review194225
---


Ship it!




Ship It!

- Colm O hEigeartaigh


On Dec. 20, 2017, 6:23 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64634/
> ---
> 
> (Updated Dec. 20, 2017, 6:23 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1929
> https://issues.apache.org/jira/browse/RANGER-1929
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently we can only edit the policy without viewing the policy. We must use 
> editing funtion of policy when only need to query the detail for policy. So 
> we should supply the function of the query detail for policy.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
> b8de5c3ba 
>   
> security-admin/src/main/webapp/scripts/views/policies/RangerPolicyConditions.js
>  PRE-CREATION 
>   security-admin/src/main/webapp/scripts/views/policies/RangerPolicyDetail.js 
> PRE-CREATION 
>   
> security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js
>  09e2e1669 
>   security-admin/src/main/webapp/styles/xa.css 22eedf644 
>   
> security-admin/src/main/webapp/templates/policies/RangerPolicyConditions_tmpl.html
>  PRE-CREATION 
>   
> security-admin/src/main/webapp/templates/policies/RangerPolicyDetail_tmpl.html
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/64634/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 64677: RANGER-1934:Optimize the init method in BaseAuditHandler class to avoid ArrayIndexOutOfBoundsException

2017-12-20 Thread Colm O hEigeartaigh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64677/#review194223
---


Ship it!




Ship It!

- Colm O hEigeartaigh


On Dec. 20, 2017, 5:54 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64677/
> ---
> 
> (Updated Dec. 20, 2017, 5:54 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1934
> https://issues.apache.org/jira/browse/RANGER-1934
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Optimize the init method in BaseAuditHandler class to avoid 
> ArrayIndexOutOfBoundsException
> The follow in the init method "   List tokens = 
> MiscUtil.toArray(propPrefix, ".");
> String finalToken = tokens.get(tokens.size() - 1);".
> in the init method we should add " if (tokens.size() > 1)" to avoid 
> ArrayIndexOutOfBoundsException.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/provider/BaseAuditHandler.java
>  b095000 
> 
> 
> Diff: https://reviews.apache.org/r/64677/diff/3/
> 
> 
> Testing
> ---
> 
> Tested it.
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 63209: RANGER-1644 changed crypto algorithm to a strong one

2017-12-20 Thread Endre Zoltan Kovacs via Review Board 


> On Dec. 7, 2017, 2:41 p.m., Velmurugan Periasamy wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java
> > Line 65 (original), 78 (patched)
> > 
> >
> > Patch fails to apply. Could you please check?
> > 
> > ```
> > $ git apply --check -v < 
> > ~/Downloads/0001-RANGER-1644-replacing-MD5-DES-with-SHA512-AES128.patch
> > Checking patch 
> > agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java...
> > error: while searching for:
> > 
> > PasswordUtils(String aPassword) {
> > String[] crypt_algo_array = null;
> > int count = 0;
> > if (aPassword != null && aPassword.contains(",")) {
> > count = StringUtils.countMatches(aPassword, ",");
> > crypt_algo_array = aPassword.split(",");
> > }
> > if (crypt_algo_array != null && crypt_algo_array.length > 
> > 1) {
> > CRYPT_ALGO = crypt_algo_array[0];
> > ENCRYPT_KEY = crypt_algo_array[1].toCharArray();
> > SALT = crypt_algo_array[2].getBytes();
> > ITERATION_COUNT = Integer.parseInt(crypt_algo_array[3]);
> > password = crypt_algo_array[4];
> > if (count > 4) {
> > for (int i = 5 ; i<=count ; i++){
> > password = password + "," + crypt_algo_array[i];
> > }
> > }
> > } else {
> > CRYPT_ALGO = DEFAULT_CRYPT_ALGO;
> > ENCRYPT_KEY = DEFAULT_ENCRYPT_KEY.toCharArray();
> > SALT = DEFAULT_SALT.getBytes();
> > ITERATION_COUNT = DEFAULT_ITERATION_COUNT;
> > password = aPassword;
> > }
> > Map env = System.getenv();
> > String encryptKeyStr = env.get("ENCRYPT_KEY");
> > if (encryptKeyStr == null) {
> > encryptKey=ENCRYPT_KEY;
> > }else{
> > encryptKey=encryptKeyStr.toCharArray();
> > }
> > String saltStr = env.get("ENCRYPT_SALT");
> > if (saltStr == null) {
> > salt = SALT;
> > }else{
> > salt=saltStr.getBytes();
> > }
> > }
> > 
> > public static String decryptPassword(String aPassword) throws 
> > IOException {
> > return new PasswordUtils(aPassword).decrypt();
> > }
> > 
> > private String decrypt() throws IOException {
> > String ret = null;
> > try {
> > byte[] decodedPassword = Base64.decode(password);
> > Cipher engine = Cipher.getInstance(CRYPT_ALGO);
> > PBEKeySpec keySpec = new PBEKeySpec(encryptKey);
> > SecretKeyFactory skf = 
> > SecretKeyFactory.getInstance(CRYPT_ALGO);
> > SecretKey key = skf.generateSecret(keySpec);
> > engine.init(Cipher.DECRYPT_MODE, key,new 
> > PBEParameterSpec(salt, ITERATION_COUNT));
> > String decrypted = new 
> > String(engine.doFinal(decodedPassword));
> > int foundAt = decrypted.indexOf(LEN_SEPARATOR_STR);
> > if (foundAt > -1) {
> > if (decrypted.length() > foundAt) {
> > ret = decrypted.substring(foundAt+1);
> > }
> > else {
> > ret = "";
> > }
> > }
> > else {
> > ret = null;
> > }
> > }
> > catch(Throwable t) {
> > LOG.error("Unable to decrypt password due to error", t);
> > throw new IOException("Unable to decrypt password due to 
> > error", t);
> > }
> > return ret;
> > }
> > }
> > 
> > error: patch failed: 
> > agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java:78
> > error: 
> > agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java:
> >  patch does not apply
> > Checking patch 
> > agents-common/src/test/java/org/apache/ranger/plugin/util/PasswordUtilsTest.java...
> > Checking patch 
> > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java...
> > error: while searching for:
> > import org.apache.poi.ss.usermodel.Workbook;
> > import org.codehaus.jettison.json.JSONException;
> > 
> > import com.google.gson.Gson;
> > 
> > @Component
> > 
> > error: patch failed: 
> >