Re: Review Request 62495: RANGER-1797:Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.82.

2017-11-30 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62495/#review192451
---


Ship it!




Ship It!

- Qiang Zhang


On Nov. 30, 2017, 1:55 p.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62495/
> ---
> 
> (Updated Nov. 30, 2017, 1:55 p.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1797
> https://issues.apache.org/jira/browse/RANGER-1797
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> [Security Vulnerability Alert] Tomcat Information leakage and remote code 
> execution vulnerabilities.
> 
> CVE ID:
> CVE-2017-12615\CVE-2017-12616
> 
> Description
> CVE-2017-12615:When running Apache Tomcat 7.0.0 to 7.0.79 on Windows with 
> HTTP PUTs enabled, it was possible to upload a JSP file to the server via a 
> specially crafted request. This JSP could then be requested and any code it 
> contained would be executed by the server.
> CVE-2017-12616:When using a VirtualDirContext with Apache Tomcat 7.0.0 to 
> 7.0.80, it was possible to use a specially crafted request, bypass security 
> constraints, or get the source code of JSPs for resources served by the 
> VirtualDirContext, thereby cased code disclosure.
> 
> Scope
> CVE-2017-12615:Apache Tomcat 7.0.0 - 7.0.79
> CVE-2017-12616:Apache Tomcat 7.0.0 - 7.0.80
> 
> Solution
> The official release of the Apache Tomcat 7.0.81 version has fixed the two 
> vulnerabilities and recommends upgrading to the latest version.
> 
> Reference
> https://tomcat.apache.org/security-7.html
> http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.81
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> 
> 
> Diffs
> -
> 
>   pom.xml 589cd6ac 
> 
> 
> Diff: https://reviews.apache.org/r/62495/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 62495: RANGER-1797:Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.82.

2017-11-30 Thread pengjianhua 


> On 十一月 30, 2017, 9:38 a.m., Vishal Suvagia wrote:
> > pom.xml
> > Line 212 (original), 212 (patched)
> > 
> >
> > @PengJianhua,
> > I used attached patch and did a build on  my local machine 
> > using mvn clean compile package.
> > After that, I ran the setup for Ranger-Admin. Then I did a 
> > ranger-admin-services start. I am getting error in catalina.out file as the 
> > Tomcat server start itself is failing(PS: attached log file on apache jira).
> > 
> > To resolve the issue I had to add a dependency for javax.annotation-api.
> > 
> > Did the attached patch work for you without adding this dependency ? If 
> > yes Kindly share how did this work for you !
> 
> pengjianhua wrote:
> Ok. I didn't add this dependency. My compiling is ok. Please delete your 
> local maven repository. Then compile the ranger project using the following 
> command:
> sudo mvn clean compile package assembly:assembly install -DskipTests
> 
> Vishal Suvagia wrote:
> Pengjianhua, the compile goes through fine. But did Ranger-Admin service 
> start using the compiled packaged bits. Are you able to access Ranger UI ?

I can access ranger UI. Your question should have nothing to do with this 
issue. If I guess good, you should be more in-depth understanding of how to use 
ranger, please refer to the manual to configure your ranger.
If you encounter problems during use, you can email me or the community.


- pengjianhua


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62495/#review192253
---


On 十一月 30, 2017, 1:55 p.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62495/
> ---
> 
> (Updated 十一月 30, 2017, 1:55 p.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1797
> https://issues.apache.org/jira/browse/RANGER-1797
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> [Security Vulnerability Alert] Tomcat Information leakage and remote code 
> execution vulnerabilities.
> 
> CVE ID:
> CVE-2017-12615\CVE-2017-12616
> 
> Description
> CVE-2017-12615:When running Apache Tomcat 7.0.0 to 7.0.79 on Windows with 
> HTTP PUTs enabled, it was possible to upload a JSP file to the server via a 
> specially crafted request. This JSP could then be requested and any code it 
> contained would be executed by the server.
> CVE-2017-12616:When using a VirtualDirContext with Apache Tomcat 7.0.0 to 
> 7.0.80, it was possible to use a specially crafted request, bypass security 
> constraints, or get the source code of JSPs for resources served by the 
> VirtualDirContext, thereby cased code disclosure.
> 
> Scope
> CVE-2017-12615:Apache Tomcat 7.0.0 - 7.0.79
> CVE-2017-12616:Apache Tomcat 7.0.0 - 7.0.80
> 
> Solution
> The official release of the Apache Tomcat 7.0.81 version has fixed the two 
> vulnerabilities and recommends upgrading to the latest version.
> 
> Reference
> https://tomcat.apache.org/security-7.html
> http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.81
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> 
> 
> Diffs
> -
> 
>   pom.xml 589cd6ac 
> 
> 
> Diff: https://reviews.apache.org/r/62495/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 64057: RANGER-1907:The solr-solrj jar is not need for hive-agent. So it should be removed from the pom.xml file of the hive-agent

2017-11-30 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64057/#review192449
---


Ship it!




Ship It!

- Qiang Zhang


On Nov. 24, 2017, 2:05 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64057/
> ---
> 
> (Updated Nov. 24, 2017, 2:05 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1907
> https://issues.apache.org/jira/browse/RANGER-1907
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The solr-solrj jar is not need for hive-agent. So it should be removed from 
> the pom.xml file of the hive-agent
> 
> 
> Diffs
> -
> 
>   hive-agent/pom.xml a2b4aa8 
> 
> 
> Diff: https://reviews.apache.org/r/64057/diff/1/
> 
> 
> Testing
> ---
> 
> I had carefully tested the ranger + hive-agent after modified the issue.
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

[jira] [Resolved] (RANGER-1910) Simplify the maven dependency management of the Yarn plugin for Ranger .solr-solrj can be removed from the pom.xml file of the plugin-yarn

2017-11-30 Thread peng.jianhua (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peng.jianhua resolved RANGER-1910.
--
   Resolution: Fixed
Fix Version/s: 1.0.0

> Simplify the maven dependency management of the Yarn plugin for Ranger 
> .solr-solrj can be removed from the pom.xml file of the plugin-yarn
> --
>
> Key: RANGER-1910
> URL: https://issues.apache.org/jira/browse/RANGER-1910
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 1.0.0, master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Minor
> Fix For: 1.0.0, master
>
> Attachments: 
> 0001-RANGER-1910-Simplify-the-maven-dependency-management.patch
>
>
> Simplify the maven dependency management of the Yarn plugin for Ranger 
> .solr-solrj can be removed from the pom.xml file of the plugin-yarn.it can 
> avoid explicitly listing some of the dependencies



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64106: RANGER-1910:Simplify the maven dependency management of the Yarn plugin for Ranger .solr-solrj can be removed from the pom.xml file of the plugin-yarn

2017-11-30 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64106/#review192448
---


Ship it!




Ship It!

- pengjianhua


On 十一月 28, 2017, 2:22 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64106/
> ---
> 
> (Updated 十一月 28, 2017, 2:22 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1910
> https://issues.apache.org/jira/browse/RANGER-1910
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Simplify the maven dependency management of the Yarn plugin for Ranger 
> .solr-solrj can be removed from the pom.xml file of the plugin-yarn.it can 
> avoid explicitly listing some of the dependencies
> 
> 
> Diffs
> -
> 
>   plugin-yarn/pom.xml 0928d81 
> 
> 
> Diff: https://reviews.apache.org/r/64106/diff/1/
> 
> 
> Testing
> ---
> 
> Tested the ranger + plugin-yarn after modified this issue.
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

[jira] [Commented] (RANGER-1837) Enhance Ranger Audit to HDFS to support ORC file format

2017-11-30 Thread Ramesh Mani (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273766#comment-16273766
 ] 

Ramesh Mani commented on RANGER-1837:
-

Thanks [~bosco] for your feedback. I shall create the new AuditFileQueue for 
handling bulk operations

> Enhance Ranger Audit to HDFS to support ORC file format
> ---
>
> Key: RANGER-1837
> URL: https://issues.apache.org/jira/browse/RANGER-1837
> Project: Ranger
>  Issue Type: Improvement
>  Components: audit
>Reporter: Kevin Risden
>Assignee: Ramesh Mani
> Attachments: 
> 0001-RANGER-1837-Enhance-Ranger-Audit-to-HDFS-to-support-.patch, 
> 0001-RANGER-1837-Enhance-Ranger-Audit-to-HDFS-to-support_001.patch, 
> AuditDataFlow.png
>
>
> My team has done some research and found that Ranger HDFS audits are:
> * Stored as JSON objects (one per line)
> * Not compressed
> This is currently very verbose and would benefit from compression since this 
> data is not frequently accessed. 
> From Bosco on the mailing list:
> {quote}You are right, currently one of the options is saving the audits in 
> HDFS itself as JSON files in one folder per day. I have loaded these JSON 
> files from the folder into Hive as compressed ORC format. The compressed 
> files in ORC were less than 10% of the original size. So, it was significant 
> decrease in size. Also, it is easier to run analytics on the Hive tables.
>  
> So, there are couple of ways of doing it.
>  
> Write an Oozie job which runs every night and loads the previous day worth 
> audit logs into ORC or other format
> Write a AuditDestination which can write into the format you want to.
>  
> Regardless which approach you take, this would be a good feature for 
> Ranger.{quote}
> http://mail-archives.apache.org/mod_mbox/ranger-user/201710.mbox/%3CCAJU9nmiYzzUUX1uDEysLAcMti4iLmX7RE%3DmN2%3DdoLaaQf87njQ%40mail.gmail.com%3E



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64189: TagSync should replace use of V1 Atlas APIs with V2 APIs for efficient tag-download from Atlas

2017-11-30 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64189/#review192406
---




pom.xml
Line 128 (original), 128 (patched)


In master branch, please refer to Apache Atlas 1.0.0-SNAPSHOT. A bunch of 
v1 classes (like IRefereceable) have been removed in Atlas; will require 
updates in Ranger tagsync. Also, new version of jackson is being used in Atlas 
- which introduces package change from org.codehaus.jackson => 
com.fasterxml.jackson.


- Madhan Neethiraj


On Nov. 30, 2017, 12:52 a.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64189/
> ---
> 
> (Updated Nov. 30, 2017, 12:52 a.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-1897
> https://issues.apache.org/jira/browse/RANGER-1897
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently tag-synchronization via REST API method uses Atlas V1 APIs, which 
> requires large number of calls from Ranger tag-sync to Atlas server. In 
> environments having large number of entities, this approach can take a long 
> time to download tags from Atlas. Use of Atlas V2 APIs would significantly 
> improve the performance.
> 
> 
> Diffs
> -
> 
>   pom.xml 589cd6a 
>   src/main/assembly/tagsync.xml 0b17151 
>   tagsync/pom.xml 74ff155 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasEntityWithTraits.java
>  77dee01 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHbaseResourceMapper.java
>  8b36a31 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHdfsResourceMapper.java
>  06bff90 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHiveResourceMapper.java
>  a359622 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasKafkaResourceMapper.java
>  09ae5d1 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
>  f007ae5 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapper.java
>  8ececdf 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapperUtil.java
>  40a639b 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasStormResourceMapper.java
>  4ed01ca 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTagSource.java
>  c382db0 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/AtlasRESTTagSource.java
>  4e0ae90 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/AtlasRESTUtil.java
>  00a101e 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntity.java
>  PRE-CREATION 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntityWithTags.java
>  PRE-CREATION 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasHdfsEntity.java
>  PRE-CREATION 
>   
> tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHbaseResourceMapper.java
>  e990c28 
>   
> tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHdfsResourceMapper.java
>  392b096 
>   
> tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHiveResourceMapper.java
>  7fde91a 
>   
> tagsync/src/test/java/org/apache/ranger/tagsync/process/TestKafkaResourceMapper.java
>  3beb82f 
> 
> 
> Diff: https://reviews.apache.org/r/64189/diff/1/
> 
> 
> Testing
> ---
> 
> Tested in local VM.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 61062: RANGER-1707 : fix hdfs traverse check

2017-11-30 Thread Abhay Kulkarni 


> On Nov. 22, 2017, 2:35 p.m., Colm O hEigeartaigh wrote:
> > Ship It!
> 
> Abhay Kulkarni wrote:
> All, 
> 
> Can we please hold on pushing this patch? I am waiting for input from 
> HDFS committers to ensure that this new HDFS authorization (Traverse 
> checking) call sequence is what is intendeded. Thanks!

HDFS dev team responded as follows.

"It looks like it is indeed a change of behaviour between 2.7 and 3.0. More 
specifically, HDFS-10997 introduced a change to FSDirectory#resolvePath, that 
when a file is accessed, this call will traversely ancestors, leading to an 
extra checkPermission() call. We don't plan to address this currently because 
this behavior sounds correct to me."

Accordingly, I have updated the patch with some modifications, and posted 
another review (https://reviews.apache.org/r/64228). Please review and comment. 
Thanks!


- Abhay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61062/#review191736
---


On Nov. 22, 2017, 12:39 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61062/
> ---
> 
> (Updated Nov. 22, 2017, 12:39 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1707
> https://issues.apache.org/jira/browse/RANGER-1707
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Fix hdfs traverse check, which problem was hidden before hdfs 2.8.0, where 
> the traverse checks are called
>  before reading and writing files, so if a policy is just about reading 
> /tmp/somedir/somefile
>  it means, that traverse should be allowed to get to that file. Adding 
> more tests to highlight the issue
> 
> 
> Diffs
> -
> 
>   hdfs-agent/pom.xml 9f6206013 
>   
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>  af4d9b5c2 
>   
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/RangerHdfsAuthorizerTest.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61062/diff/3/
> 
> 
> Testing
> ---
> 
> Tested locally
> https://travis-ci.org/gzsombor/ranger/builds/256331500
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Review Request 64228: Traverse check in RangerHdfsAuthorizer works incorrectly

2017-11-30 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64228/
---

Review request for ranger, Zsombor Gegesy, Madhan Neethiraj, Ramesh Mani, and 
Velmurugan Periasamy.


Bugs: RANGER-1707
https://issues.apache.org/jira/browse/RANGER-1707


Repository: ranger


Description
---

Traversal check in RangerHdfsAuthorizer works incorrectly, when it is asked for 
access to /a/b/c.txt, it only checks that if there are a policy which grants 
EXEC to /a/b, but if it there aren't any, then it doesn't check, if there is a 
policy which grants READ, WRITE or EXEC to /a/b/c.txt explicitly, which would 
mean, that the path is accessible to the user.
This hasn't noticed by the current unit tests, because HDFS before 2.8.0 
doesn't called the traversal check before reading or writing a file, however it 
will cause problem with 2.8.0, where FSDirectory.resolvePath will perform a 
mandatory traversal check.

This patch is based on the patch submitted for review 
(https://reviews.apache.org/r/61062/) with following modifications.
1. If traversal check (check for EXECUTE on the parent/ancestor if resource is 
a file) does not fail with explicit DENY by Ranger Authorizer, then it is 
presumed to have succeeded without any further checks and no audit record 
created. If it fails with DENY, then the authorization fails and an audit 
record is created.
2. Test policies in hdfs-policies.json and test cases 
(RangerHdfsAuthorizerTest) are modified to test for explicit DENY case.


Diffs
-

  hdfs-agent/pom.xml 87ba777 
  
hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
 af4d9b5 
  
hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/RangerHdfsAuthorizerTest.java
 PRE-CREATION 
  hdfs-agent/src/test/resources/hdfs-policies.json 056231f 


Diff: https://reviews.apache.org/r/64228/diff/1/


Testing
---

Unit tested with HDFS versions 2.7.1 and 3.0.0.


Thanks,

Abhay Kulkarni

[jira] [Comment Edited] (RANGER-1488) Create Ranger plugin for gaiandb

2017-11-30 Thread Nigel Jones (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272877#comment-16272877
 ] 

Nigel Jones edited comment on RANGER-1488 at 11/30/17 4:16 PM:
---

Added some notes on what I've seen so far as I run through the gaianDB code. I 
will look at subtask 8 .. but this was needed as prep. Will continue next week

The test data source used was 
https://cwiki.apache.org/confluence/display/ATLAS/Example+of+virtualizing+MySQL+employee+data+via+gaianDB
 - there's further gaiandb info via the atlas wiki


was (Author: jonesn):
Added some notes on what I've seen so far as I run through the gaianDB code. I 
will look at subtask 8 .. but this was needed as prep. Will continue next week

> Create Ranger plugin for gaiandb
> 
>
> Key: RANGER-1488
> URL: https://issues.apache.org/jira/browse/RANGER-1488
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Nigel Jones
>Assignee: Nigel Jones
>  Labels: VirtualDataConnector
> Attachments: GaianDebuggingResearch.docx
>
>
> GaianDB is a distributed, federated database built on Apache Derby.
> The documentation is at 
> https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for 
> the source.
> As part of a Virtual Connector Project based on Atlas we are using gaianDB to 
> provide a virtualization layer. We need to control access to underlying 
> resources and will be building a Ranger plugin for gaiandb to support this. 
> GaianDB already has support for a form of policy plugin which allows the SQL 
> to be intercepted.
> It is unclear if this code will sit
>  - external to ranger/atlas ie in the gaianDB sources
>  - as a sample in the Atlas project (so that we collate what is needed for 
> the virtual connector project)
>  - as a sample, or component in the Ranger project
> However it seems helpful to describe the proposal here and get 
> feedback/dialogue in the ranger community :-)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1488) Create Ranger plugin for gaiandb

2017-11-30 Thread Nigel Jones (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272877#comment-16272877
 ] 

Nigel Jones commented on RANGER-1488:
-

Added some notes on what I've seen so far as I run through the gaianDB code. I 
will look at subtask 8 .. but this was needed as prep. Will continue next week

> Create Ranger plugin for gaiandb
> 
>
> Key: RANGER-1488
> URL: https://issues.apache.org/jira/browse/RANGER-1488
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Nigel Jones
>Assignee: Nigel Jones
>  Labels: VirtualDataConnector
> Attachments: GaianDebuggingResearch.docx
>
>
> GaianDB is a distributed, federated database built on Apache Derby.
> The documentation is at 
> https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for 
> the source.
> As part of a Virtual Connector Project based on Atlas we are using gaianDB to 
> provide a virtualization layer. We need to control access to underlying 
> resources and will be building a Ranger plugin for gaiandb to support this. 
> GaianDB already has support for a form of policy plugin which allows the SQL 
> to be intercepted.
> It is unclear if this code will sit
>  - external to ranger/atlas ie in the gaianDB sources
>  - as a sample in the Atlas project (so that we collate what is needed for 
> the virtual connector project)
>  - as a sample, or component in the Ranger project
> However it seems helpful to describe the proposal here and get 
> feedback/dialogue in the ranger community :-)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (RANGER-1488) Create Ranger plugin for gaiandb

2017-11-30 Thread Nigel Jones (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nigel Jones updated RANGER-1488:

Attachment: GaianDebuggingResearch.docx

Some rough notes of working through the gaiandb code

> Create Ranger plugin for gaiandb
> 
>
> Key: RANGER-1488
> URL: https://issues.apache.org/jira/browse/RANGER-1488
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Nigel Jones
>Assignee: Nigel Jones
>  Labels: VirtualDataConnector
> Attachments: GaianDebuggingResearch.docx
>
>
> GaianDB is a distributed, federated database built on Apache Derby.
> The documentation is at 
> https://github.com/gaiandb/gaiandb/blob/master/README.md - navigate up for 
> the source.
> As part of a Virtual Connector Project based on Atlas we are using gaianDB to 
> provide a virtualization layer. We need to control access to underlying 
> resources and will be building a Ranger plugin for gaiandb to support this. 
> GaianDB already has support for a form of policy plugin which allows the SQL 
> to be intercepted.
> It is unclear if this code will sit
>  - external to ranger/atlas ie in the gaianDB sources
>  - as a sample in the Atlas project (so that we collate what is needed for 
> the virtual connector project)
>  - as a sample, or component in the Ranger project
> However it seems helpful to describe the proposal here and get 
> feedback/dialogue in the ranger community :-)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 62495: RANGER-1797:Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.82.

2017-11-30 Thread Vishal Suvagia via Review Board 


> On Nov. 30, 2017, 9:38 a.m., Vishal Suvagia wrote:
> > pom.xml
> > Line 212 (original), 212 (patched)
> > 
> >
> > @PengJianhua,
> > I used attached patch and did a build on  my local machine 
> > using mvn clean compile package.
> > After that, I ran the setup for Ranger-Admin. Then I did a 
> > ranger-admin-services start. I am getting error in catalina.out file as the 
> > Tomcat server start itself is failing(PS: attached log file on apache jira).
> > 
> > To resolve the issue I had to add a dependency for javax.annotation-api.
> > 
> > Did the attached patch work for you without adding this dependency ? If 
> > yes Kindly share how did this work for you !
> 
> pengjianhua wrote:
> Ok. I didn't add this dependency. My compiling is ok. Please delete your 
> local maven repository. Then compile the ranger project using the following 
> command:
> sudo mvn clean compile package assembly:assembly install -DskipTests

Pengjianhua, the compile goes through fine. But did Ranger-Admin service start 
using the compiled packaged bits. Are you able to access Ranger UI ?


- Vishal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62495/#review192253
---


On Nov. 30, 2017, 1:55 p.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62495/
> ---
> 
> (Updated Nov. 30, 2017, 1:55 p.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1797
> https://issues.apache.org/jira/browse/RANGER-1797
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> [Security Vulnerability Alert] Tomcat Information leakage and remote code 
> execution vulnerabilities.
> 
> CVE ID:
> CVE-2017-12615\CVE-2017-12616
> 
> Description
> CVE-2017-12615:When running Apache Tomcat 7.0.0 to 7.0.79 on Windows with 
> HTTP PUTs enabled, it was possible to upload a JSP file to the server via a 
> specially crafted request. This JSP could then be requested and any code it 
> contained would be executed by the server.
> CVE-2017-12616:When using a VirtualDirContext with Apache Tomcat 7.0.0 to 
> 7.0.80, it was possible to use a specially crafted request, bypass security 
> constraints, or get the source code of JSPs for resources served by the 
> VirtualDirContext, thereby cased code disclosure.
> 
> Scope
> CVE-2017-12615:Apache Tomcat 7.0.0 - 7.0.79
> CVE-2017-12616:Apache Tomcat 7.0.0 - 7.0.80
> 
> Solution
> The official release of the Apache Tomcat 7.0.81 version has fixed the two 
> vulnerabilities and recommends upgrading to the latest version.
> 
> Reference
> https://tomcat.apache.org/security-7.html
> http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.81
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> 
> 
> Diffs
> -
> 
>   pom.xml 589cd6ac 
> 
> 
> Diff: https://reviews.apache.org/r/62495/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 62495: RANGER-1797:Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.82.

2017-11-30 Thread pengjianhua 


> On 十月 10, 2017, 5:19 a.m., bhavik patel wrote:
> > @pengjianhua : Any updates on this?
> 
> pengjianhua wrote:
> I am testing SSL/Kerberos for Ranger KMS.
> 
> pengjianhua wrote:
> I tested the patch. The Java 1.8 is required. That is to say users must 
> upgrade jdk to 1.8 above.
> 
> pengjianhua wrote:
> I had verified SSL/Kerberos for admin\kms. And I will merge the issue.
> 
> Colm O hEigeartaigh wrote:
> Why is Java 1.8 required?

The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
equal to true.
That is that the java 1.8 required when user only sets db_ssl_enabled equal to 
true.


- pengjianhua


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62495/#review187494
---


On 十一月 30, 2017, 1:55 p.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62495/
> ---
> 
> (Updated 十一月 30, 2017, 1:55 p.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1797
> https://issues.apache.org/jira/browse/RANGER-1797
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> [Security Vulnerability Alert] Tomcat Information leakage and remote code 
> execution vulnerabilities.
> 
> CVE ID:
> CVE-2017-12615\CVE-2017-12616
> 
> Description
> CVE-2017-12615:When running Apache Tomcat 7.0.0 to 7.0.79 on Windows with 
> HTTP PUTs enabled, it was possible to upload a JSP file to the server via a 
> specially crafted request. This JSP could then be requested and any code it 
> contained would be executed by the server.
> CVE-2017-12616:When using a VirtualDirContext with Apache Tomcat 7.0.0 to 
> 7.0.80, it was possible to use a specially crafted request, bypass security 
> constraints, or get the source code of JSPs for resources served by the 
> VirtualDirContext, thereby cased code disclosure.
> 
> Scope
> CVE-2017-12615:Apache Tomcat 7.0.0 - 7.0.79
> CVE-2017-12616:Apache Tomcat 7.0.0 - 7.0.80
> 
> Solution
> The official release of the Apache Tomcat 7.0.81 version has fixed the two 
> vulnerabilities and recommends upgrading to the latest version.
> 
> Reference
> https://tomcat.apache.org/security-7.html
> http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.81
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> 
> 
> Diffs
> -
> 
>   pom.xml 589cd6ac 
> 
> 
> Diff: https://reviews.apache.org/r/62495/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 62495: RANGER-1797:Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.82.

2017-11-30 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62495/
---

(Updated 十一月 30, 2017, 1:55 p.m.)


Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
Neethiraj, Velmurugan Periasamy, and Qiang Zhang.


Bugs: RANGER-1797
https://issues.apache.org/jira/browse/RANGER-1797


Repository: ranger


Description
---

[Security Vulnerability Alert] Tomcat Information leakage and remote code 
execution vulnerabilities.

CVE ID:
CVE-2017-12615\CVE-2017-12616

Description
CVE-2017-12615:When running Apache Tomcat 7.0.0 to 7.0.79 on Windows with HTTP 
PUTs enabled, it was possible to upload a JSP file to the server via a 
specially crafted request. This JSP could then be requested and any code it 
contained would be executed by the server.
CVE-2017-12616:When using a VirtualDirContext with Apache Tomcat 7.0.0 to 
7.0.80, it was possible to use a specially crafted request, bypass security 
constraints, or get the source code of JSPs for resources served by the 
VirtualDirContext, thereby cased code disclosure.

Scope
CVE-2017-12615:Apache Tomcat 7.0.0 - 7.0.79
CVE-2017-12616:Apache Tomcat 7.0.0 - 7.0.80

Solution
The official release of the Apache Tomcat 7.0.81 version has fixed the two 
vulnerabilities and recommends upgrading to the latest version.

Reference
https://tomcat.apache.org/security-7.html
http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.81
https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82


Diffs (updated)
-

  pom.xml 589cd6ac 


Diff: https://reviews.apache.org/r/62495/diff/3/

Changes: https://reviews.apache.org/r/62495/diff/2-3/


Testing
---


Thanks,

pengjianhua

Re: Review Request 62495: RANGER-1797:Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.82.

2017-11-30 Thread pengjianhua 


> On 十一月 30, 2017, 9:38 a.m., Vishal Suvagia wrote:
> > pom.xml
> > Line 212 (original), 212 (patched)
> > 
> >
> > @PengJianhua,
> > I used attached patch and did a build on  my local machine 
> > using mvn clean compile package.
> > After that, I ran the setup for Ranger-Admin. Then I did a 
> > ranger-admin-services start. I am getting error in catalina.out file as the 
> > Tomcat server start itself is failing(PS: attached log file on apache jira).
> > 
> > To resolve the issue I had to add a dependency for javax.annotation-api.
> > 
> > Did the attached patch work for you without adding this dependency ? If 
> > yes Kindly share how did this work for you !

Ok. I didn't add this dependency. My compiling is ok. Please delete your local 
maven repository. Then compile the ranger project using the following command:
sudo mvn clean compile package assembly:assembly install -DskipTests


- pengjianhua


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62495/#review192253
---


On 十月 10, 2017, 7:01 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62495/
> ---
> 
> (Updated 十月 10, 2017, 7:01 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1797
> https://issues.apache.org/jira/browse/RANGER-1797
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> [Security Vulnerability Alert] Tomcat Information leakage and remote code 
> execution vulnerabilities.
> 
> CVE ID:
> CVE-2017-12615\CVE-2017-12616
> 
> Description
> CVE-2017-12615:When running Apache Tomcat 7.0.0 to 7.0.79 on Windows with 
> HTTP PUTs enabled, it was possible to upload a JSP file to the server via a 
> specially crafted request. This JSP could then be requested and any code it 
> contained would be executed by the server.
> CVE-2017-12616:When using a VirtualDirContext with Apache Tomcat 7.0.0 to 
> 7.0.80, it was possible to use a specially crafted request, bypass security 
> constraints, or get the source code of JSPs for resources served by the 
> VirtualDirContext, thereby cased code disclosure.
> 
> Scope
> CVE-2017-12615:Apache Tomcat 7.0.0 - 7.0.79
> CVE-2017-12616:Apache Tomcat 7.0.0 - 7.0.80
> 
> Solution
> The official release of the Apache Tomcat 7.0.81 version has fixed the two 
> vulnerabilities and recommends upgrading to the latest version.
> 
> Reference
> https://tomcat.apache.org/security-7.html
> http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.81
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> 
> 
> Diffs
> -
> 
>   pom.xml 3958014c 
> 
> 
> Diff: https://reviews.apache.org/r/62495/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 64051: RANGER-1906 - Simplify Atlas plugin dependency management

2017-11-30 Thread Colm O hEigeartaigh 


> On Nov. 30, 2017, 6:52 a.m., Mehul Parikh wrote:
> > @Colm : Can you please confirm if Audit to HDFS and Audit to Solr are 
> > working after removal of these dependencies?

Hi Mehul,

I haven't actually removed any dependencies with this patch as such - the 
distribution jars are exactly the same. The Solr jar is bundled via the 
agents-common dependency, and the Atlas plugin takes the Hadoop jars from Atlas 
itself instead of via the lib directory in the distribution.


- Colm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64051/#review192242
---


On Nov. 28, 2017, 11:44 a.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64051/
> ---
> 
> (Updated Nov. 28, 2017, 11:44 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1906
> https://issues.apache.org/jira/browse/RANGER-1906
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> This task is to simplify the dependency management for the Atlas plugin. In 
> particular, the hadoop/solr dependencies should just be imported via the 
> ranger-plugin-commons dependency.
> 
> No changes are made to the resulting jars for the Atlas distribution.
> 
> 
> Diffs
> -
> 
>   plugin-atlas/pom.xml 957b4ce3 
>   ranger-atlas-plugin-shim/pom.xml a207d16b 
>   src/main/assembly/plugin-atlas.xml fd988116 
> 
> 
> Diff: https://reviews.apache.org/r/64051/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

[jira] [Updated] (RANGER-1797) Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.82.

2017-11-30 Thread Vishal Suvagia (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vishal Suvagia updated RANGER-1797:
---
Attachment: catalina.out

Attaching [^catalina.out], as per info shared on review request.

> Tomcat Security Vulnerability Alert. The version of the tomcat for ranger 
> should upgrade to 7.0.82.
> ---
>
> Key: RANGER-1797
> URL: https://issues.apache.org/jira/browse/RANGER-1797
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 1.0.0, master
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>  Labels: patch
> Attachments: 
> 0001-RANGER-1797-Tomcat-Security-Vulnerability-Alert.-The.patch, catalina.out
>
>
> 【Security Vulnerability Alert】Tomcat Information leakage and remote code 
> execution vulnerabilities.
> CVE ID:
> {code}
> CVE-2017-12615\CVE-2017-12616
> {code}
> Description
> {code}
> CVE-2017-12615:When running Apache Tomcat 7.0.0 to 7.0.79 on Windows with 
> HTTP PUTs enabled, it was possible to upload a JSP file to the server via a 
> specially crafted request. This JSP could then be requested and any code it 
> contained would be executed by the server.
> CVE-2017-12616:When using a VirtualDirContext with Apache Tomcat 7.0.0 to 
> 7.0.80, it was possible to use a specially crafted request, bypass security 
> constraints, or get the source code of JSPs for resources served by the 
> VirtualDirContext, thereby cased code disclosure.
> {code}
> Scope
> {code}
> CVE-2017-12615:Apache Tomcat 7.0.0 - 7.0.79
> CVE-2017-12616:Apache Tomcat 7.0.0 - 7.0.80
> {code}
> Solution
> {code}
> The official release of the Apache Tomcat 7.0.81 version has fixed the two 
> vulnerabilities and recommends upgrading to the latest version.
> {code}
> Reference
> {code}
> https://tomcat.apache.org/security-7.html
> http://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.81
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)